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Abstract 

In this paper, we consider the AWGN channel with a power constraint called the (cr, p)-power 
constraint, which is motivated by energy harvesting communication systems. Given a codeword, 
the constraint imposes a limit oi a + kp on the total power of any fc > 1 consecutive transmitted 
symbols. Such a channel has infinite memory and evaluating its exact capacity is a difhcult 
task. Gonsequently, we establish an n-letter capacity expression and seek bounds for the same. 

We obtain a lower bound on capacity by considering the volume of 5„(cr, p) C M", which is the 
set of all length n sequences satisfying the (cr, p)-power constraints. For a noise power of v, we 
obtain an upper bound on capacity by considering the volume of Sn{o,p) 0 i3„(vJzE), which 
is the Minkowski sum of 5„(cr, p) and the n-dimensional Euclidean ball of radius ^/nv. We 
analyze this bound using a result from convex geometry known as Steiner’s formula, which gives 
the volume of this Minkowski sum in terms of the intrinsic volumes of 5 „((j, p). We show that 
as the dimension n increases, the logarithm of the sequence of intrinsic volumes of p)} 

converges to a limit function under an appropriate scaling. The upper bound on capacity is then 
expressed in terms of this limit function. We derive the asymptotic capacity in the low and high 
noise regime for the (ct, p)-power constrained AWGN channel, with strengthened results for the 
special case of cr = 0, which is the amplitude constrained AWGN channel. 

Keywords: Additive white Gaussian noise, energy harvesting, (cr, p)-power constraint, Minkowski 
sum, Steiner’s formula. Shannon capacity, intrinsic volumes. 


1 Introduction 

The additive white Gaussian noise (AWGN) channel is one of the most basic channel models studied 
in information theory. This channel is represented by a sequence of channel inputs denoted by Xi, 
and an input-independent additive noise Zi. The noise variables Zi are assumed to be independent 
and identically distributed as AA(0, u). The channel output Yi is given by 


Yi = Xi + Zi for i > 1. (1) 

The Shannon capacity this channel is infinite in case there are no constraints on the channel inputs 
Xf, however, practical considerations always constrain the input in some manner. These input con¬ 
straints are often defined in terms of the power of the input. For a channel input (xi,X 2 ,. •. ,Xn), 
the most common power constraints encountered are; 
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(AP): An average power constraint of P > 0, which says that 


xf < nP. 

i=l 

(PP): A peak power constraint of A > 0, which says that 

\xi\ < A for all 1 < i < n. 

(APP): An average and peak power constraint, consisting of (AP) and (PP) simultaneously. 

The AWGN channel with the (AP) constraint was first analyzed by Shannon [1]. Shannon showed 
that the capacity C for this constraint is given by 

C= sup /(X;y) = ^log (l +-V (2) 

E[X2]<P ^ V ^ J 

and the supremum is attained when X ~ AA(0, P). Here capacity is defined in the usual sense, due 
to Shannon. See Section 2 for a precise definition. 

Compared to the (AP) constraint, fewer results exist about the (PP) constrained AWGN. The 
AWGN channel with the (PP) constraints was first analyzed by Smith [2]. Smith showed that the 
channel capacity C in this case is given by 

C= sup I{X-,Y). (3) 

|X|<A 

Unlike the (AP) case, the supremum in equation (3) does not have a closed form expression. Using 
tools from complex analysis. Smith established that the optimal input distribution attaining the 
supremum in equation (3) is discrete, and is supported on a finite number on points in the interval 
[—A, A]. He proposed an algorithm to numerically evaluate this optimal distribution, and thus 
the capacity. Smith also analyzed the (APP) constrained AWGN channel and derived similar 
results. In a related problem, Shamai & Bar-David [3] studied the quadrature Gaussian channel 
with (APP) constraints, and extended Smith’s techniques to establish analogous capacity results 
for the same. 

Our work in this paper is primarily concerned with a power constraint, which we call a {a, p)- 
power constraint, defined as follows: 

Definition. Let a, p > 0. A codeword {xi,X 2 , ■ ■ ■, Xn) is said to satisfy a (u, p)-power constraint if 

i 

x] < O' + {I — k)p ,\/0<k<l<n. (4) 

j=k+l 

These constraints are motivated by energy harvesting communication systems, a research area 
which has seen a surge of interest in recent years. Energy harvesting (EH) is a process by which 
energy derived from an external source is captured, stored, and harnessed for applications. For 
example, harvested energy in the form of solar, thermal, or kinetic energy is converted into electrical 
energy using photoelectric, thermoelectric, or piezoelectric materials, and is used to power electronic 
devices. Energy which is harvested is generally present as ambient background and is free. EH 
devices are efficient, cheap, and require low maintenance, making them an attractive alternative 
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Figure 1: Block diagram of a general energy harvesting communication system 


to battery-powered devices. The problem of communicating over a noisy channel using harvested 
energy is encountered in a prominent application of EH: wireless sensor networks. Typically, sensor 
nodes used in such networks are battery-powered and thus have finite lifetimes. Since EH sensor 
nodes are capable of harvesting energy for their functioning, they have potentially infinite lifetimes 
and thereby have many advantages over their battery-powered counterparts [4]. 

We can model communication scenarios like the “EH sensor node” via a general energy harvest¬ 
ing communication system shown in Eigure 1. Here, the transmitter is capable of harvesting energy, 
and uses it to transmit a codeword X”, corresponding to a message W. The transmitter has a 
battery to store the excess unutilized energy, which can be used for transmission later. The amount 
of energy harvested in time slot i, denoted by Ei, can be modeled as a stochastic process. The 
process Ei, along with the battery capacity, determines the power constraints that the codeword 
X” has to satisfy. This codeword is transmitted over a noisy channel, and the receiver decodes 
W using the channel output E”'. A natural channel to study in this setting is the classical addi¬ 
tive Gaussian noise (AWGN) channel. Suppose we have a channel model as in Eigure 2; namely, 
an AWGN channel with an energy harvesting transmitter which harvests a constant p amonnt of 
energy per time slot, and which has a battery of capacity a attached to it. 


W- 


1 


X(0,X) 


Encoder — 

- 'Xi^Yi 


Decoder 


■W 


Eigure 2: (a, /9)-power constrained AWGN channel 


To understand the power constraints imposed on a transmitted codeword (xi,X 2 , ■ ■ ■, Xn) in this 
scenario, we define a state ai, for each i > 0 as 


(To = cr, and (Tj+i = min(cr, ai + p — xf) . (5) 

Erom the energy harvesting viewpoint, we can think of the state ai as the charge in the battery at 
time i before transmitting Xi, assuming the battery started out fully charged at time 0. Denote by 
Sn{a,p) C M"- the set 

Sn{a, p) = {x” GM":cJi>0, V 0 < i < nj. (6) 

In words, the set Sn{a,p) consists of sequences {xi,X 2 , ■ ■ ■ ,Xn) such that at no point during 
its transmission, is there a need to overdraw the battery. Thus, this set is precisely the set of 
all possible length n sequences which the transmitter is capable of transmitting. Telescoping the 
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minimum in equation (5), we get that for all i > 0, 

( * 

(Tj+i = min cj, a + p - xf, • • • , a + ip - 

\ i=i 

Using the condition cTj > 0 for all z, we obtain another characterization of 5n(cr, p): 

i 

Sniu, p) = {x” G M" : ^ < fj + (/ — k)p , V 0 < A: < / < n}, 

j=fc+i 



(7) 


( 8 ) 


which is exactly the (cr,/9)-power constraint defined in equation (4). It is interesting to note that 
such (u, p)-constraints were originally introduced by Cruz [5, 6] in connection with the study of 
packet-switched networks. We first look at the (tr, /9)-power constraint for the extreme cases; namely, 
(7 = 0 and a = oo. 


No battery: 

Suppose that the battery capacity a is 0; i.e., unused energy in a time slot cannot be stored for 
future transmissions. We can easily check that for a transmitted codeword (xi, X 2 ,..., x^), the power 
constraints 

xj < p, for every 1 < z < n (9) 

are necessary and sufficient to satisfy the inequalities in (4). Thus, the case of a = 0 is simply the 
(PP) constraint of y^. 

Infinite battery: 

Consider the case where the battery capacity is now infinite, so that any unused energy can be 
saved for future transmissions. We assume that the battery is initially empty, but we can equally 
well assume it to start with any finite amount of energy in this scenario. The constraints imposed 
on a transmitted codeword (xi, X 2 ,..., x,i) are 

k 

xf < kp, for every 1 < i < n. (10) 

i=l 

It was shown by Ozel &: Ulukus [7] that the strategy of initially saving energy and then using a 
Gaussian codebook achieves capacity, which is ^ log (l + ^). In fact, [7] considers not just constant 
Ei, but a more general case of i.i.d. Ei. 

Finite battery: 

An examination of equations (4) and (5) reveals that the energy constraint on the n + 1-th symbol 
Xn+i, depends on the entire history of symbols transmitted up to time n. This infinite memory 
makes the exact calculation of channel capacity under these constraints a difficult task. For some 
recent work on discrete channels with finite batteries, we refer the reader to Tutuncuoglu et. al. 
[8, 9] and Mao &: Hassibi [10]. An alternative model of an AWGN channel with a finite battery was 
also considered by Dong et. al. [11], where the authors established approximate capacity results 
for the same. 
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In this paper, we will primarily focus on getting bounds on the channel capacity of an AWGN 
channel with (a, /3)-power constraints. Our work can be broadly divided into two parts; the first 
part deals with getting a lower bound, and the second part with getting an upper bound. The 
approach for both these parts relies on analyzing the geometric properties of the sets Sn{cr,p). In 
what follows, we briefly describe our results. 

1.1 Lower bound on capacity 

We obtain a lower bound on the channel capacity in terms of the volume of 5n(cr, p). More precisely, 
we define v{a,p) to be the exponential growth rate of volume of the family {5n((T, p)}: 

v{a,p):= lim - log Vol(5n(fT, p)), (11) 

n^oo n 

where the limit can be shown to exist by subadditivity. Our first result is Theorem 3.2 in Section 
3, which contains a lower bound on the channel capacity: 

Theorem 3.2. The capacity C of an AWGN channel with a {a, p)-power constraints and noise 
power 1 / satisfies 

Having obtained this lower bound on C, it is natural to study the dependence of v{a, p) on its 
arguments. Theorem 4.1 in Section 4 establishes the following: 

Theorem 4.1. For a fixed p, v{a,p) is a monotonically increasing, continuous, and concave func¬ 
tion of a over [0,oo), with its range being [log 2y^, ^ log 27rep). 

In Section 5, we describe a numerical method to find v{a, p) for any value of the pair (cr, p). This 
calculated value can be used to compare the lower and upper bounds in Theorem 3.2 for different 
values of a for a fixed p. From the energy-harvesting perspective, this comparison indicates the 
benefit that a finite battery of capacity a has on the channel capacity. With this we conclude the 
first part of the paper. 


1.2 Upper bound on capacity 

The upper bound on capacity in (12) is not satisfactory as it does not depend on a. Our approach 
to deriving an improved upper bound on capacity also involves a volume calculation. However, the 
improved upper bound is not in terms of the volume of Sn{cr,p), but in terms of the volume of 
the Minkowski sum of Sn{<y,p) and a “noise ball.” Let Bn{\/nv) be the Euclidean ball of radius 
y/nv. The Minkowski sum of Sn{(y, p) and (also called the parallel body of 5 „((t, p) at a 

distance \/ni'), is defined by 

Sn{cr, p) © BniVw) = {x" + z" I x"- G Sn{(r, p), z" G BniVnn)} . (13) 


In Section 6, we prove the following upper bound on capacity: 


Theorem 6.1. 

power V satisfies 


The capacity C of an AWGN channel with a {a, p)-power constraint and noise 


C < 


1, Vol{Snicr, p) ® Bn{y/n{u e) )) 

hm hm sup - log-- 

e^o+ n^oo n Vol[BnWnn)) 


(14) 
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This motivates us to define a function I : [0, oo) —>■ M, giving the growth rate of the volume of 
the parallel body as follows: 

l{v) := limsup — log Vol(5„(cr, p) ©)). (15) 

n^oo 

The upper bound can be restated as 

C < limsup 

e^0+ 

To study the properties of £{■), we use the following result from convex geometry called Steiner’s 
formula: 

Theorem 6.2. Let C M"" be a compact convex set and let C M"" be the unit ball. Denote by 
fj.j{Kn) the j-th intrinsic volume Kn, and by ej the volume of Bj. Then for t>0, 

n 

Vol{Kn © tBn) = Y, Tn-jiKn)ejtT (17) 

1=0 

Intrinsic volumes are a fundamental concept in convex and integral geometry. They describe 
the global characteristics of a set, including the volume, surface area, mean width, and the Euler 
characteristic. For more details, we refer the reader to Schneider [12] and section 14.2 of Schneider 
& Weil [13]. 

In Section 7, we focus on the a = 0 case for two reasons. Firstly, intrinsic volumes are notoriously 
hard to compute for arbitrary convex bodies. But when a = 0, the set 5„(cr, p) is simply the cube 
[—^/p, ^/p]'^■ The intrinsic volumes of a cube are well known in a closed form, which permits an 
explicit evaluation of In his paper. Smith [2] numerically evaluated and plotted the capacity 
of a (PP) constrained AWGN channel. Based on the plots. Smith noted that as i/ —)• 0, the channel 
capacity seemed to satisfy 

C = log 2T — - log 27rez^ + o(l), (18) 

where the o(l) terms goes to 0 as —)• 0. He gave an intuitive explanation for this phenomenon as 
follows: Let X be the amplitude-constrained input, let Z ~ J\f{0, u) be the noise, and let Y be the 
channel output. Then for a small noise power v, h(Y) ~ h{X), and 

C = sup7(A;F) 

= sup h{Y) — h{Y\X) 

X 

^ sup h{X) -h{Y\X) 

X 

= log 2A — - log 27rez^. 

Note that the crux of this argument is that when the noise power is small, supj^ ^(P) ~ sup^ = 
log 2A. This argument can me made rigorous by establishing 


£{i' + e) — - log 27reu 


(16) 


lim sup h{X + Z) 
I X 


log2A = 0. 


(19) 


Recall that our upper bound on capacity is C < limsup£_,.Q^ {£{u + e) — 2log27rez/} . Since 
7(0) = log2j4, the continuity of 7 at 0 would lead to asymptotic upper bound which agrees with 
Smith’s intuition. The following theorems provide our main result for the case of cr = 0: 
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Theorem 7.1. The function £(i^) is continuous on [ 0 , 00 ). For u > 0, we can explicitly compute 
£{u) via the expression 

‘2.TTP1J 

£[v) = H{d*) + (1 - r ) log2A + - log —, (20) 

where H is the binary entropy function, and 9* G (0,1) satisfies 

(1 - _ 2^2 
9*^ TTU 

Theorem 7.6. The capacity C of an AWGN channel with an amplitude constraint of A, and with 
noise power v, satisfies the following: 

1. When the noise power —>■ 0, capacity C is given by 

C = log 2^ — ^ log27rez^ + 0(z^3). 


2. When the noise power v — >■ 00 , capacity C is given by 

v6 




where a = Aj 


We also establish a general entropy upper bound, which does not require the noise Z to be 
Gaussian: 


Theorem 7.7. Let A,v > 0. Let X and Z be random variables satisfying |X| < A a.s. and 
Var{Z) < n. Then 

h{X + Z)<£{v). (21) 


Finally, in Section 8 we turn to the case of cr > 0. Unlike the cr = 0 case, the intrinsic volumes 
of Sn{(r, p) are not known in a closed form. For n > 1, we let {p,n{0), ■ , Pnin)} be the intrinsic 

volumes of Sn{cr, p). The sequence of intrinsic volumes {pn{-)}n'>i forms a sub-convolutive sequence 
(analyzed in Appendix F). Convergence properties of such sequences can be effectively studied using 
large deviation techniques; in particular, the Gartner-Ellis theorem [14]. These convergence results 
for intrinsic volumes can be used in conjunction with Steiner’s formula to establish results about 
£ and the asymptotic capacity of a {a, /9)-constrained channel in the low noise regime. Our main 
results here are: 

Theorem 8.1. Define £{v) as 

£{v) = limsup — log Vol{Sn{(y, p) © Bn{'/rw )). (22) 

n^oo ^ 

For n > 1, define Gn : M —?• M and : M —?• M as 

” . G (f) 

Gn{t) = log"^ Pn{j)e^\ and g^it) = — — . (23) 

j=0 

Define A to be the pointwise limit of the sequence of functions {pn}, which we show exists. Let A* 
be the convex conjugate of A. Then the following hold: 
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1. (.{v) is continuous on [ 0 , 00 ). 


2. For 1 / > 0, 


i{n) 


sup 

6»e[o,i] 


_A*(l-0) + ^log 


2'Kev 


(24) 


Theorem 8.10. The capacity C of an AWGN channel with {a, p)-power constraints and noise 
power V satisfies the following: 

1. When the noise power n ^ 0, capacity C is given by 


C = v(a, /o) - 2 log 27rei/ + e(n), 


where e(-) is a function such that = 0 . 

2. When noise power 1 / —)> 00 , capacity C is given by 

1 //9N 2 


C = 


1 (P\^ , 1 (ffi 

6 \v. 



2 Channel Capacity 

We define channel capacity as per the usual convention [15]: 

Definition 1. A (2”^,n) code for the AWGN channel with a (cr,/ 9 )-power constraint consists of 
the following: 

1 . A set of messages { 1 , 2 ,..., 2 l-"^-l} 

2 . An encoding function / : { 1 , 2 ,... , 2 l-"'^J} — Snicjp), yielding codewords /(I),... ,/( 2 l-"'^-l) 

3. A decoding function 5 : M"' —>• { 1 , 2 ,..., } 

A rate R is said to be achievable if there exists a sequence of (2”^, n) codes such the that probability 
of decoding error diminishes to 0 as n —>■ 00 . The capacity of this channel is the supremum of all 
achievable rates. 

Shannon’s formula for channel capacity 


C = sup 
X 


(25) 


is valid if the channel is memoryless. 
expression to 

C = 


For a channel with memory, one can often generalize this 


lim sup-/(X”;y") , 
n^oo Ti J 


(26) 


but this formula does not always hold. Dobrushin [16] showed that channel capacity is given 
by formula (26) for a class of channels called information stable channels. Checking information 
stability for specific channels can be quite challenging. Fortunately, in the case of a (cr, / 9 )-power 
constrained AWGN channel, we can establish formnla (26) without having to check for information 
stability. We prove the following theorem: 







Theorem 2.1. For n £ N, let Tn be the set of all probability distributions supported on Snicr,p)- 
The capacity C of a {a, p)-power-constrained scalar AWGN channel is given by 

C= lim - sup (27) 

Proof. Let be a positive integer. Without loss of generality, we can assume that coding is done 
for block lengths which are multiples N, say nN. For codes over such blocks, we relax the {a, p) 
constraints as follows. For every transmitted codeword - ,XnN)i each consecutive block 

of N symbols has to lie in 5Ar((T, p); i.e.. 


{xkN+i,XkN+2, ■■■■,X(^k+i)N) e 57 v((t, p), for 0 < /c < n - 1. (28) 

Note that this is indeed a relaxation because a codeword satisfying the constraint (28) is not guar¬ 
anteed to satisfy the (u, p)-constraints but any codeword satisfying the (cr, p)-constraints necessarily 
satisfies the constraint (28). The capacity of this channel Cm can be written as 

Cn= sup I{X^-Y^). (29) 


This capacity provides an upper bound to A^C for any choice of N. Thus, we have the bound 


C < inf 

N 


Cm 

N ■ 


(30) 


To show that inf at Cm/X is liniM Cm/X, we first note that 

T /-y M-\-N, \ ^ T ( yM, yM \ i j( y M-\-N, yM-\-N \ 

^ 5 ^ ^ J \^M+l ’ ^M+1 ) • 

Taking the supremum on both sides with Pxm+n ranging over Fm+n-, 

(31) 

(32) 

(33) 


Cm^n < 


sup 

PxM+N 


(^IiXf-,Y^^) + I{X 


M-l-1 ’ ^ M-i-l 


(a) 

< 


sup /(Xf;y/ 

PxM (x^)&Tm 


-\- sup I{X 

PxN 


Af. 

1 I 1 1 


= Cm + Cm- 


Here (a) follows due to the containment Fm+n ^ Fm x Fn- This calculation shows that {Cat} 
is a sub-additive sequence. Applying Fekete’s lemma [17] we conclude that limArCAr/Ai exists and 
equals inf at Cat/A", and thereby establish the upper bound 


C < lim 

TV^OO 


cv 

X ■ 


(34) 


We now show that C is lower bounded by WtumCm/X. Given any xi,X 2 ,-- - ,x„ G Sm, the 
concatenated sequence xi • • • x„ need not always satisfy the (a, p) power constraints. However, if 
we append fc = [^] zeros to each x* and then concatenate them, the n{X -\- k) length string so 
formed lies in S^f^M+k)- This is because transmitting [^] zeros after each x* ensures that the state, 
as defined in equation (5), returns to a before the transmission of Xj+i begins. Let us define a new 
set 

5Ar = {x'^+'^:xf G5^,xj^+^ = 0}. 
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The earlier discussion implies that 


Sn X • • • Sn C Sn(N+k)- 

^ V 

n times 


(35) 


Equation (35) implies that any block coding scheme which uses symbols from Sjsr is also a valid 
coding scheme under the {a, p) power constraints. The achievable rate for such a scheme can 
therefore provide a lower bound to C. This achievable rate is simply Cat, as the final k transmissions 
in each symbol carry no information. Thus the per transmission achievable rate is and we 

get that 

(36) 


for all N. Taking the limit as N 


^-N + k' 
oo, we arrive at the bound 


C > lim 
N^OO N 


The containment (34), together with the inequality (37), completes the proof. 


(37) 


□ 


3 Lower-bounding capacity 

Coding with the {a, p) constraints can be thought of as trying to fit the largest number of centers 
of noise balls in Sn, such that the noise balls are asymptotically approximately disjoint. One might 
therefore hope to get a packing based upper bound on capacity through the volume of Sn ■ We shall 
show that the volume of Sn surprisingly yields a neat lower bound on capacity. 

Let Vn{cr,p) denote the volume of Sn{(T,p)- We look at the exponential growth rate of this 
volume defined by 

logl4(cr, p) 


v{a, p) ;= lim 


n 


(38) 


Our first lemma is to establish the existence of the limit in the definition of u(ct, p). 
Lemma 3.1. lim^^oo (q'-p) exists. 

Proof. The containment Sm+nicr, p) C Sm{cr, p) x Sn{cr, p) gives 

km+n(f^)P) ^ ^m{,^ 1 p')i^n{,^: p') ^ 


which implies 


log Vm+n {cr,p) < log Vyn (cT, /o) + log 14 (cr, /o). 


This shows that log 14(a, p) is a sub-additive sequence, and by Fekete’s Lemma, the limit lim„_^oo 
exists and is equal to in4 (-^vtiich may a priori be —oo). 


log v„(a-,p) 

n 


□ 


Theorem 3.2. The capacity C of an AWGN channel with (cr, p)-power constraints and noise power 
V satisfies 


1 

2 


log 


1 + 


^2v(a,p) 

27Teiy 


<C < 




(39) 
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Proof. Clearly, C is upper bounded by the capacity for the a = oo case (with zero initial battery 
condition), which by [7] is ^ log(l + ^). 

Let the noise Z ~ A7(0, v). To prove the lower bound, recall the capacity expression in Theorem 

2 . 1 : 


C= lim - sup (40) 

= lim - sup h{Y^) - h{Z^) (41) 

= lim — sup h{Y"') — - log 2TTeu (42) 


Thus, calculating capacity requires maximizing the output differential entropy h{Y"‘). Using Shan¬ 
non’s entropy power inequality, we have 

2h(Y'^) 2h{X") 2h{Z") 

gn >6" +6". (43) 


Thus, 


2h(Y") 

sup e " > sup -^ + 2Trei> 

PX-n (x" ) e J'n Px'i (a:" ) 

2 log 

= e ^ + zTrez^. 


Taking logarithms on both sides and letting n tend to infinity, we have 

lim sup ^ > - log + 27rei^) 

^^°°Pxn(x")G^n n 2 V y 

which, combined with equation (42) concludes the proof. 


(44) 

□ 


4 Properties of v(a, p) 

We can readily see that v{a,p) is monotonically increasing in both of its arguments. With a little 
more effort, we can also establish the following simple bounds for v{cr, p): 

log 2^ < v{a, p) < log yj2Tiep. (45) 

To show the lower bound from inequality (45), observe that if x” is such that for every 1 < z < n, 

la^il < y/P, 

then the (cr, /9)-constraints are satisfied. Thus, the cube [—y^, \/p]" of volume (2y^)"' lies inside 
the set Sn (<7, p ), giving the lower bound 

v{a,p) > log 2^. 

For the upper bound, we use the “total power” constraint, 

x\ + xl + ... + < a + np, 
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which implies that Sn{cr,p) C Bn{\/cr + np), where Bn{\/cr + np) is the Euclidean ball of radius 
The volume 14,(c, p) of the set 5n(cr, p) is bounded above by the volume of Bn{^a' + np), 
which gives 


v{(T,p) < lim - log ( T. /I , + 

n^oon \ r (I + 1) 

1 


2 ~ (^)) 


n^oo 

= - log 2Txep. 


Note that when p = 0, then v{a, 0) = —oo for any value of a. Henceforth, we assume p > 0. When 
cr = 0, the set Snicr, p) degenerates to the cube [—■\/p, ^/p]'^ , which has the volume growth rate 
exponent of log 2^4- It is clear that when cr > 0, the set Sn{cr,p) contains the cube [—^/p,^/p]^, 
implying that 

Vn{<T,p) > ( 2 ^)’". 

However, this does not immediately imply that v{a,p) > log2y4- The following theorem is the 
main result of this section, where we show that such a strict inequality holds, and also prove some 
other properties of the function v{a, p): 

Theorem 4.1. For a fixed p, v{a,p) is a monotonically increasing, continuous, and concave func¬ 
tion of a ^ [0,oo), with its range being [log2y4, 5log27rep). 

Proof of Theorem 4-1. Theorem 4.1 relies on several lemmas. We state the lemmas here and defer 
their proofs to Appendix A. We first show that it is enough to prove the theorem for p = 1: 

Lemma 4.2 (Proof in Appendix A.l). Let vi{a) = v{a,l). Then v{a,p) depends on vfia/p) 
according to 

v{(y,p) = log^ + vi{a/p). (46) 

Thus, a different value of p leads to a function v{a,p) which is essentially vi{a) shifted by a 
constant. Therefore, if ui(cr) is monotonically increasing, continuous, and concave, so is v{a,p) for 
any other value of p > 0. In Lemmas 4.3 and 4.4, we establish that vi{a) is a continuous and 
concave function on [0,oo): 

Lemma 4.3 (Proof in Appendix A. 2). The function vi{a) is continuous on [0,oo). 

Lemma 4.4 (Proof in Appendix A. 3). The function ui(cr) is concave on [0,oo). 

To finish the proof, we need to show that the limiting value of ui((t) as u —)• oo is | log27re. It 
is useful to define a quantity, which we call burstiness of a sequence, as follows: Let An denote the 
the n-dimensional ball of radius -v/n; i.e., 


An := < x” : ^ xj <n 


2 = 1 


Fix E An‘ We associate a burstiness to each such sequence, defined by 


a(x ) := max 

0<k<l<n 




( 47 ) 


\i=k-\-l 
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Let 


Anicr) = {x^ G An : < cr}. 

Notice that Anicr) C Sn{(x, 1). We have .4^(0) = [—1, 1]” and An(n — 1) = An- As a increases from 
0 to n — 1, An{<y) increases from the cube to the entire sphere. We have the following lemma: 

Lemma 4.5 (Proof in Appendix A. 4). If there exists a sequence cr(n) such that 

lim —log VoUAnicrin))) = -log27re , and (48a) 

n^QO n 2 

lim = 0, (48b) 

n^oo Tl 

then limo-_>.oo ui((t) = ^ log27re. 

Note that the natural choice which satisfies condition (48a) is (j{n) = n — 1, but this does not 
satisfy condition (48b). To complete the proof, we show that (T{n) = c^/n for a suitable constant c 
satishes both conditions of Lemma 4.5, and establish the following result: 

Lemma 4.6 (Proof in Appendix A. 5). limo—>.oo ^) ~ \ log27re. 

This completes the proof of Theorem 4.1. □ 


5 Numerical method to compute v{a,p) 

In this section, we briefly discuss the numerical evaluation of v{a, p). This discussion is nontechnical 
and for all the technical details justifying the numerical method, we refer the reader to Appendix B. 


Numerical computation of v{a, p) is enabled by exploiting the idea of state as defined in equation 
(5). However for ease of analysis and implementation, we dehne the state slightly differently. Given 
(xi,..., Xn) G Sn{cr, 1), define 


(Tn if CTn ^ U, 

fJn-l + 1 - X^ if cr„ = cr. 


(49) 


The state (fn is a sum of two terms: an, which is the amount of charge in the battery at time n, 
and the amount of energy wasted at time n due to the limited battery capacity. Note that energy 
is wasted only when an = a; i.e., when the battery becomes full. Setting pQ = a, equation (49) can 
also be written as 


T,_i + 1 - xi 
a+ l-xl 


if f^n—l ^ 
if > (X- 


(50) 


Consider the function : Sn{a, 1) —>■ M, dehned by <l)„(xi,..., Xn) = </>n- Thus, ^n maps a point 
in Snia, 1) to its state at time n, as defined in equations (49) and (50). Let \n be the Lebesgue 
measure restricted to Sn{a, 1). The function <1>„ induces a measure on M, which we call Vn- As 
0 < <?in < <7 + 1 for all x'^ G Sn{a, 1), we see that the measure i^n is supported on [0,cr + 1], giving 


i/n([0, 0- + 1]) = Vol(5n(cr, 1)). 


Suppose i^n is absolutely continuous with respect to the Lebesgue measure on M; this implies 
existence of a density /„ corresponding to Vn, which satisfies 


Vo1(5„(ct, 1)) 



fn{x)dx. 


(51) 
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Given the state 4>n < cr, the symbol Xn+i is constrained to lie in [—y/4>n + Ij V'/'n + !]• Furthermore, 
given (pm the symbol Xn+i has the Lebesgue measure restricted to this set. Similarly, for pn > <7, 
the conditional measure of Xn+i is the Lebesgue measure restricted to [—y/a + 1, y/a + 1]. Using 
equation (50), we can find a relation between the measures Vn and t'n+i as follows; For p £ [0, (7+1], 


Fn+l{(t>) '■= ^'n+l((-OO,0]) 

.0-1 /-+l/„(x) 


+ 

+ 


r<p-^ r 

J x=0 J t- 

/ 

-1 Jt-- 


dtdx 


dtdx 


dtdx 


=0 y/i 

fn{x) 

'x=<f>—i Jt=x—{<f>—i) y/t 

r+' fn{x) 

J x=a Jt=(T—{4>—X) 

/■ 0-1 

= / 2fn{x)[y/x + l]dx 
Jx=Q 

+ [ 2fnix)[y/x + 1- y/x- {(p- l)]dx 
J X = (l)—1 

+ / 2fn{x)[y/a + I - y/a - {(p- l)]dx. 


Differentiating Fn+i, we obtain 

fn+l{(p) = 


-1 y/x-{p- 1) 
Define the integral operator A as follows: 




fn{x) 


yja-{p-l) 


dx. 


A{x, t) 


V++T-+ 

1 

\/cr+l—t 

0 


if 0 < X < cj and 0 < t < x + 1, 
ifcj<x<(T + l and 0 < t < cr + 1, 
otherwise. 


(52) 


(53) 


(54) 


(55) 


(56) 


We can express equation (55) in another form, 

fn+i{t) = J A{x,t)fn{x)dx, (57) 

denoted by fn+i = ^(/n)- Iterating this relation, we obtain 


fn+l=A^h. 


(58) 


Our interest is in ui(cr), which by equations (51) and (58) is 

xi(cr) = lim — log [ A^~^fi{x)dx. (59) 

n-roo n Jq 

It seems natural to expect this limit to equal the largest eigenvalue of A. Our approach to finding 
the largest eigenvalue is to discretize A] let hn = and let An be an (n + 1) x (re + l)-matrix 
such that 

Anii,j) = hn X A{{i - l)hn,{j - l)hn) for l<i,i<re + l. 
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Figure 3: Graph of vi{a) obtained numerically 


We can approximate the largest eigenvalue of the matrix An using standard methods and expect 
this value to tend to the largest eigenvalue of as n becomes large. 

Figure 3 shows the plot of vi{a) obtained using the numerical procedure. Note that as a 
becomes large, vi{a) tends to the limit ^ log27re in a concave manner, as per Theorem 4.1. 

We are now in a position to plot the bounds on capacity derived in Theorem 3.2. Figure 4 shows 
a plot of the lower and upper bounds for a fixed value of p (= 1) and for different values of the noise 
power L'. Note that even for relatively small values of a, the volume based lower bound on capacity 
is close to the upper bound, which we recall is the channel capacity when a = oo. Thus, a small 
battery leads to significant gains in the capacity of a {a, p)-power constrained AWGN channel. 


6 Upper-bounding capacity 


Theorem 3.2 states that t log (l + upper-bounds the channel capacity. This bound is not entirely 
satisfactory since it is independent of the value of a. Furthermore, Figure 4 indicates that the 
lower bound and the upper bound do not converge asymptotically: as —)• 0, the lower bound 
is u((T, p) — 5 log27rei^ + O(z^) and the upper bound is ^ log ^ which differ by 0(1). This 

implies that either the upper bound, or the lower bound, or both, are loose in the low-noise regime. 
It is natural to expect the upper bound to be loose, since it disregards the effects of a finite a on 
capacity. To obtain some insight on the low-noise capacity, it is useful to think of coding with the 
(cj,/?)-constraints as trying to fit the largest number of centers of noise balls in 5 „((t, p), such that 
the noise balls are asymptotically approximately disjoint. As the noise power u decreases, so does 
the size of the noise balls, and one can imagine a very efficient packing of these small balls so that 
they occupy almost all the available space. The total number of balls one can pack is then roughly 
given by 


# of balls 


Vol(5n(q', p)) 
Vol(Noise ball) ’ 


(60) 
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Figure 4: Capacity lower bounds for u = 0,1,5, and 10, and the upper bound, from Theorem 3.2 
plotted versus log(l/i^) 


so the capacity is roughly 


— log # of balls 
n 


_ Vol(5n((T, p) 

n Vol(Noise ball) 

~ P)-\ log 2'Kev. 


(61) 

(62) 


We can make the statement in equation (62) rigorous, as follows: 

Theorem 6.1. Let Bn{y/rw) he the n-dimensional Euclidean ball of radius y/rw. The Minkowski 
sum of Sn{(y,p) and Bn{\dnu) is the set 

5n(cr, p) © BniVrw) := {x"" + z” | x” G 5 „(ct, p), G Bni^/ny)}. (63) 

The capacity C of an AWGN channel with a {a, p)-power constraint and noise power u satisfies 


C < lim limsup — log Vol{Sn{cr, p) © Bn{\/ n{v + e) ))- \og2'Ken. (64) 

n-s-oo n 2 

Proof. For n G N, let Fn be the set of all probability distributions supported on Sn{o',p). From 
Theorem 2.1, we know that the capacity C is given by 

(7= lim - sup (65) 


Let px^-ix'^) £ J^n- Denote Sn{o',p) © Bn{y/n{u + e) ) by Cn, where e > 0, and let 5n '■= 
P ^ Cn) ■ By the law of large numbers, we have 6n —)• 0. Let y be the indicator variable for the 
event {Y^ ^ Cn}- Then 


h{Y^) = H{5n) + SnhiY'^lx = 1) + dnhiY'^lx = 0) 

< H{Sn) + dnlogYoliCn) + SnHY^lx = 0 ). ( 66 ) 
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where a = 1 — a. Since < a + np with probability 1, we have the following bound on power 

of y"; 


E[\\Y^f] = E[\\X^f] + E[\\Z^f] <a + np + niy. 
This translates to the bound 


E[\\Y^f 


X = 0] < 


n{p + v + a/n) 


so 


2 Ofi 

Substituting into inequality (66) and dividing by n gives 

h{Y^) H{6n) ^ logVol(C'n) 5n, 2TTe{p + v + a/n) 

- < -h On -h — log--. 

n n n 2 On 

Since this holds for any choice of px^ £ Xn, we obtain 

sup V°1(C„) ^ 2^e(p + p + a/n) ^ 

Tl 2 U-ti 

Taking the limsup in n, we arrive at 

logVo^C^) logVol(5n((T,p) ©5„(v/n(z^ + e) )) 

iimsup sup —h{Y ) < limsup-= iimsup-. 

n^oo px‘^^2^n ^ n^oo ^ n^oo U 


Taking the limit as e —>■ 0+ and noting that capacity is hm„_>.oo supp^^gj-^ ^/i(y”) — ^ log27rei/, we 
arrive at the bound in expression (64). □ 

To simplify notation, define t : [0, oo) —>■ M as 

i{v) := limsup — logVol(5„(cr,p) © i?„(\/nr')). (67) 

n^oo U- 


We can restate the upper bound in Theorem 6.1 as 

C < lim + e) — 1 log 2'Keu. 
£—> 0 + 2 


( 68 ) 


If ^ happens to be continuous at v, we can drop the e from inequality (68) to obtain a simplified 
expression 


C < i{i') — - log 27reu. (69) 

Note that £(0) = v{a,p). The continuity of ^ at = 0 can be used to rigorously establish the 
asymptotic capacity expression in equation (62). These continuity properties will be established 
later in this paper. 

The upper bound expression involves the volume of the Minkowski sum of Snio', p) with a ball. 
We state here a result from convex geometry called Steiner’s formula [18], which gives an expression 
for the volume of such a Minkowski sum: 
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Theorem 6.2 (Steiner’s formula). Let Kn C M"" he a compact convex set and let Bn C M”" he the 
unit hall. Denote hy p,j{Kn) the j-th intrinsic volume Kn, and hy Cj the volume of Bj. Then for 
t > 0, 

n 

Vol{Kn © tBn) = y^^lJ,n-j{Kn)ejtT (70) 

j=0 

Steiner’s formula states that the volume of 5„(fT, p)® Bn{\/^) depends not only on the volumes 
of these sets, but also on the intrinsic volumes of 5„((T, p). Intrinsic volumes are notoriously hard 
to compute even for simple enough sets such as polytopes [18]. So it is optimistic to expect a closed 
form expression for the intrinsic volumes of 5^(0", p). Furthermore, the sets {Sn{cr,p)} evolve with 
the dimension n, and to compute the volume via Steiner’s formula it is necessary to keep track of 
how the intrinsic volumes of these sets evolve with n. 

As mentioned earlier, the case of cr = 0 is the amplitude-constrained Gaussian noise channel, the 
capacity of which was numerically evaluated by Smith [2]. In the following section, we concentrate 
on evaluating the upper bound for this special case. 

7 The case of cr = 0 

To simplify notation, we denote A := yf~p in this section. We consider the scalar Gaussian noise 
channel with noise power n and an input amplitude constraint of A. Let the capacity of this channel 
be C. Recall that the function is defined as 

i{i') = limsup - log Vol([—A, A]"'© i?„(-v/n7)), (71) 

n—^oo 

and the upper bound on channel capacity is given by 

C < lim i(u + e) — - log 27rei/. 

£->0+ 2 

The main result of this section is as follows: 

Theorem 7.1. The function l{v) is continuous on [0,cx)). For u > 0, we can explicitly compute 
l{v) via the expression 

‘2,'kpij 

£{u) = H{d*) + (1 - r) log2A + - log —, (72) 

where H is the binary entropy function, and 9* G (0,1) is the unique solution to 

(1 - r)2 _ 2A2 

9*^ TTU 

Proof of Theorem 7.1. The proof of Theorem 7.1 relies on a number of lemmas. Here we shall 
merely state the lemmas and defer their proofs to Appendix C. 

We first prove a lemma, which makes it possible to replace lim sup by lim in the expression of 
l{v) given in equation (71). 

Lemma 7.2 (Proof in Appendix C.l). For all u >0, the limit 

lim - log Fo/([—A, A]” © BniVrw)) 

n^oo u 

exists and is finite and equals (.{v), as defined in equation (71). 
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by 


The special case of Steiner’s formula (70) when is the cube [—A, A]'^ and t = y/w is given 


Vol([-A,^]”' © BniV^)) = ^ (2^)” ^ej{y/wy 

j=o 


where ej is the volume of the j-dimensional unit ball. Replacing €j in equation (73), 


/n\ 7rJ/2 

Vol(|-A,A]"®B„(VS:)) = ^ 

r(n + 1) 


E 


i/2 


J^o + 1 ) 


Letting 9 = -, we rewrite the term inside the summation as 


r(n + 1) 


TT 


ne/2 


r(n(l -9) + l)T{ne + 1) r(n0/2 + 1) 
For u > 0, define /^(0) as follows: 


(2^)n(l-0)(^)n0_ 


1 


/nW = -log 


r(n + 1) 


TT 


ne/2 


n ^ \T{n{l-9) + l)r{ne + l)T{ne/2 + l) 


(2^)n(l-0)(^) 


nO 


= - log 
n 


r(n + l)n’^®/2 


(73) 


(74) 


i2Ar-\V^y. (75) 


(76) 


(77) 


r(n(l -6) + l)r{ne + l)r(n0/2 + 1) 


+ (1 - 0) log 2^ + 0 log + - log TT. 


(78) 


Note that /)((0) is defined for all n G N, for all 9 G [0,1], and for all > 0. Using this notation, we 
can rewrite the volume as 

n 

Yol{[-A,Ay © BniV^)) = (79) 

j=0 

We argue that since the volume is a sum of n + 1 terms, the exponential growth rate of the volume 
is determined by the growth rate of the largest term amongst these n + 1 terms. To be precise, we 
define 

9n = argmcixf^{j/n), (80) 

j/n 

and prove the following lemma; 

Lemma 7.3 (Proof in Appendix C.2). The limit lim^^oo fni^n) exists and equals 

The next few lemmas aim to identify the limit of f^{9n)- We first show that the functions fni') 
converge uniformly to a limit function 

Lemma 7.4 (Proof in Appendix C.3). The sequence of functions {fn}’^f=i converges uniformly for 
all 9 G [0,1] to a function given by 

fj Q.ttpu 

r{9) = H{9) + {l-9) log 2A + - log —, (81) 

where H{9) = —9\og9 — {1 — 9) log(l — 9) is the binary entropy function. 
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With this uniform convergence in hand, we show that the limit of can be expressed as 

follows: 

Lemma 7.5 (Proof in Appendix C.4). We claim that 

lim fn0n) = max/"'((9), (82) 

n^oo 9 

and therefore 

^{v) = max/'^(0). (83) 

9 

We are now in a position to prove the continuity of (.{v). Fix a t'o > 0, and let e > 0 be given. 
Choose a (5 > 0 such that for all u ^ {vq — 5, vq + 5), 

\\r-r\\oo<e. 

We can verify from equation (81) that picking such a 5 is indeed possible. This implies 

|supr(0)-supr°(0)| <e. (84) 

e e 


Using Lemma 7.5, this implies 


\i{v) -£{no)\ < e, 


which establishes continuity of i at all points i^q > 0. 


(85) 


To show continuity at 0, we first explicitly evaluate (■{v). Let 0*{v) 
Lemma 7.5, we have (.{v) = Recall the expression for 


argmaxe/^(0). Using 


f) ^.TTPiy 

ne) = H{d) + {I-6) log2A + - log —. 
Differentiating with respect to 9, 

^f''{9) = log + log \/T - log 2A + ^ log 27re - ^ log 9. 

Setting the derivative equal to 0 gives 

1 — 0 1 loS 6 1 

log—^-h log ^/^9 - log 2A + -log27re- - --log6» = 0. 

Simplifying this and removing the logarithms, we arrive at 

(1 - 0)2 _ 2A2 
9^ TTiy 


( 86 ) 


(87) 


( 88 ) 


(89) 


The function tends to +00 as 0 —)• 0+, and equals 0 when 0 = 1. Thus, equation (89) has 

at least one solution in the interval (0,1). We can easily check that is strictly decreasing in 

(0,1), and thus this solution must be unique. The optimal 9*(r') satishes the cubic equation (89), 
and we can see that 

Iim0*(j/) = O. (90) 

i/—>0 
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Using equations (89) and (90), we have 


lim „ , 


2A^ 

IT 


Thus, 


Yiui Uv) = Ymi H{e*{v)) + (1 - 0*(i/)) log 2^ + log 
i^-5-O 2 0*[v) 

(a) , . , 1 27rez^ 

= log 2A + hm- - log ■ 


i/—^b 2 


9*{v) 


= log2A 

=m, 


(91) 

(92) 

(93) 

(94) 

(95) 


where in (a) we used equation (90), and in (6) we used equation (91). This shows that i is continuous 
over [0, oo), and concludes the proof of Theorem 7.1. □ 

The above bound can also be used to prove an asymptotic capacity result. We prove the 
following theorem: 

Theorem 7.6. The capacity C of an AWGN channel with an amplitude constraint of A, and with 
noise power u, satisfies the following: 

1. When the noise power —)• 0, capacity C is given by 


C = log2^ — - log27rez^ + 0{h'3). 


2. ITTien the noise power v —>■ oo, capacity C is given by 

^ a® 5a® in, 

2 4 6 24 ^ 

where a = A/^/v. 

Proof of Theorem 7.6. Note that all the logarithms in this proof are assumed to be to base e. 
1. Using the lower bound in Theorem 3.2, 


C > log I 1 + 


1 


{2Af 

27rei' 


= log 2A — - log 2Treu + log ^1 + 
= log 2A — ^ log 27rei' + 0{i'). 


27rez.^ 

(^ 


For the upper bound, we have 


lim (.(v) = log 2A + lim 


^TTPU 

H{e*)-e*\og2A + -\og — 


/I ^ 

= log 2A + lim -(1 - 0*) log(l -9*) + — \og^ + — log 


(96) 

(97) 

(98) 

(99) 

( 100 ) 
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1/0 

Let c = { 2 ^) ■ Using equation (91), we can check that as i/ —)• 0, 

-(1 - r) log(l - r) = 


9* 7re 


+ 5£^) .'/3 + „(,l/3). 


This gives the following asymptotic upper bound as —)■ 0: 

C < log 2A — ^ log 27rez^ + 

From equations (98) and (101), our claim follows. 


( 101 ) 


2. As noted by Smith [2], for large the optimal input distribution is discrete, and is supported 
equally on the two points —A and +A. The output Y is then distributed as 

'{y + Ar 


Y r^priy) = ^exp 




2v 


Capacity is then given by 


( 102 ) 


C = h{pY) — - log27rei^. 

The entropy term h{pY) can be manipulated as in [19] to arrive at 


1 2 
Hpy) = X log 27rei/ + -( 

2 Y[2Tr)a 


-«V2 


f 


-y^l2o? 


(103) 


cosh(y) lncosh(?/)(iy, (104) 


where a = Aj^. Let 

2 -y^ 

f{a) = / cosh(y) In (cosh(?/))(i?/. 

vSvr Jo 

We consider the Taylor series expansion of cosh(y) In (cosh(7/)) at y = 0, and arrive at 


,2 yA yO 


cosh(y) log (cosh(y)) = — + — + ^ -h — -h O (y^) . 

B V 2 6 720 630 ^ 


(105) 


Using the following definite integral expression. 




and substituting, we obtain 


„9 


a" ii, 

^(“) = T + T + 48 + T + °(“ ) 


Thus, 


(106) 


(107) 


h{pY) = - log 27rez/ + 


I o? a® 

-It + t + 48 + t + °'“ 


10 ' 


a 


4 6 

a a 


1-Y +Y-^ + o(« 


1 


o? 


v6 


= L log 27rez^ Y— -«-5c^ 0{q^^) 

2 ^ 2 4 6 24 ^ ^ 


(108) 

( 109 ) 


22 










Capacity is therefore given by 


C 


o? a® 5a® 

T ~ T "6 ^ 


+ 0(a^°). 


This establishes the claim. Shannon [1] had proved that capacity at high noise for the peak 
power constrained (by AWGN channels is essentially the same as that of an average power 
constrained (by AWGN; i.e., 


C 


-log(l + 


- log(l + a^) 
o? a® a® , 


10 \ 


It is interesting to note that the first three terms of this approximation agrees with the actual 
capacity. 


□ 


We can use Theorem 7.1 to numerically evaluate Q*{y') and plot the corresponding upper bound 
from Theorem 6.1. Figure 5 shows the resulting plot. Note that the upper bound from Theorem 
3.2 is not asymptotically tight in the low-noise regime, but the new upper bound is asymptotically 
tight. 



Figure 5: For the AWGN with an amplitude constraint of 1, the new upper bound and the lower 
bound converge asymptotically as —>■ 0 


In Theorem 7.1, we essentially carried out a volume computation which answered the question: 
How does the volume of the Minkowski sum of a cube and a ball grow? The upper bound on 
capacity is then a consequence of the following facts: 

1. The channel capacity depends on the maximum output entropy h{Y'^). 

2. The random variable is (almost entirely) supported on the sum of a cube and a ball. 

3. The entropy of Y^ is bounded from above by the logarithm of the volume of its (almost) 
support. 
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Intuitively, points 2 and 3 should not depend on Z being Gaussian, but only on being almost 
entirely supported on Bn{^\frw\ We make this intuition precise in the following theorem: 

Theorem 7.7 (Proof in Appendix C.5). Let A,v > 0. Let X and Z he random variables satisfying 
\X\ < A a.s. and Var(Z} < u. Then 


h{X + Z)<l{v), (110) 

where i{u) is as defined in equation (71). 

By Theorem 7.7, we can assert that the capacity C of any channel with input amplitude con¬ 
strained by A and with an additive noise Z with power at most i' is bounded from above according 
to 

C= sup I{X;X + Z)<i{u)-h{Z). (Ill) 

|X|<A 

Noting that Var(Y) < + u, we also have the upper bound 

C < ^loglTreA + A^) — h{Z). (112) 

giving 

C < min — h{Z), - log 27re{u + A^) — h{Z)^ . (113) 

From Figure 5, it is interesting to note that there for large values of v, the bound in inequality 
(112) is better, whereas for small values of v, the bound in inequality (111) is better. Both of these 
bounds are asymptotically tight as —>■ oo, but only inequality (111) is tight for z/ —)• 0. 

8 The case of cr > 0 

In this section, our aim is to parallel the upper-bounding technique used in Section 7 and obtain 
analogues of Theorem 7.1 and Theorem 7.6, when a is strictly greater than 0. When ct > 0, the 
set Sniu, p) is no longer an easily identifiable set like the n-dimensional cube from Section 7. In 
particular, the intrinsic volumes of Sn{cr,p) do not have a closed form expression. Despite this 
difficulty, we shall see that it is still possible to obtain results similar to those in Section 7. 

Our main result in this section is the following: 

Theorem 8.1. Define Av) as 

i{v) =limsup-log Vol{Sn{cr, p) 0) Bniy/rw )). (114) 

n^oo ^ 

For n>l, denote the intrinsic volumes of Sn{cr, p) by Pn{i) for 0 < f < n and define Gn : M —)> M 
and : M —M as 

Gn{t) = log V/rn(j)e^*, gn{t) = ( 115 ) 

n 

j=0 

Define A to be the pointwise limit of the sequence of functions {gn}, which we will show exists. Let 
A* be the convex conjugate of A. Then the following hold: 

1. (.{v) is continuous on [0, oo). 
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2. For 1 / > 0, 


(116) 


i{u) 


sup 

6»e[o,i] 


_A*(l-0) + ^log 


2Trei' 


Proof of Theorem 8.1. Note that for the statement of Theorem 8.1 to make sense, several results 
need to be established. We establish these in the Lemmas 8.2 and 8.3, where we prove the following: 

Lemma 8.2 (Proof in Appendix D.l). For all n > 1, the set Sn{o', p) is a eonvex set, and therefore 
it has well defined intrinsic volumes {pn{i)}f=o- 

Lemma 8.3 (Proof in Appendix D.2). The following results hold: 

1. The functions {pn} converge pointwise to a function A(t) : M —)• M given by 


A{t) := lim gn{t). 

n^oo 


(117) 


2. The convex eonjugate of A, denoted by A*, has its domain the set [0,1]. 

By Lemma 8.2, we can use Steiner’s formula for the convex set Sn{cr,p) to get 

n 

Vol(5n(cr,p) © BniVrw )) = ^//„(n - j)ej^/rw^. (118) 

1=0 

Dehne the functions a„(0) and bl^{6) for 6 G [0,1] as follows. The function an{9) is obtained by 
linearly interpolating the values of an{j/n), where the value of an{j/n) is given by: 

an (-)=-log pn{n - j) for 0 < j < n. (119) 

\n J n 

The function hn{6) is given by 

Dehne /^ : [0,1] —)■ M as 

rn{e) -.= nn,e) = an{e) + b'f{e). ( 121 ) 

With this notation, we can rewrite equation (118) as 

n 

\o\{Sn{cj,p) © Bn{V^)) = (122) 

1=0 

Just as in the proof of Theorem 7.1, we want to establish the convergence of fn(') to some function 
/^(•). Proving the convergence of &(((•) is not hard, but proving the convergence of o„(-) requires 
the application of Lemmas 8.4 and 8.5 given below. In Lemma 8.4 we establish the following: 

Lemma 8.4 (Proof in Appendix D.3). For each n, the following holds: 

1. The function an{-) is concave. 

2. The function bn{-) is concave. 

3. The function fni') is concave. 
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In Lemma 8.5, we show that the intrinsic volumes of {5,i(cj,/?)} satisfy a large deviations-type 
result, detailed below. 

Lemma 8.5 (Proof in Appendix D.4). Define a sequence of measures supported on [0,1] by 


^n/n ( “ ) •= for 0 < j < n. 


(123) 


The following bounds hold: 


1. Let / C M 6e a closed set. The family of measures {fJ-n/n} satisfies the large deviation upper 
bound 

(124) 


limsup — log/U„/„(/) < — inf A*(x). 
n^oo n xei 

2. Let F CM. be an open set. The family of measures {fJ^n/n} satisfies the large deviations lower 
bound 

liminf — log/i„/„(F) > — inf A*(x). (125) 

r)_ n ' X^F 


n^oo 71 


Using the concavity and large deviations-type convergence from the two previous lemmas, we 
now prove the convergence of {/^} in the following lemma. 

Lemma 8.6 (Proof in Appendix D.5). The following convergence results hold: 

1. The sequence of functions {a^} converges uniformly to —A*(l — 6) on [0,1]. 

2. The sequence of functions {bf^} converges uniformly to the function ^ log on [0,1]. 

3. The sequence of functions {/^} converges uniformly to a function on the interval [0,1], 
where f^ is given by 

r(9) = -A*(l-e) + ^log^. 

We are now in a position to express i(u) in terms of the limit function f’^. Let 

On = arg max^y„/^(j'/n). 


In Lemma 8.7 we prove the following: 

Lemma 8.7 (Proof in Appendix D.6). The following equality holds: 

lim fniOn) = maxf^{e). 

n^oo 0 

Lemma 8.8 (Proof in Appendix D.7). The following equality holds: 

lim fni^n) = K^)-, 


and therefore 


e{u) = snpr{e). 

g 


(126) 


(127) 

(128) 
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Part 2 of Theorem 8.1 follows from Lemma 8.8. We now concentrate on proving the continuity 
of We hrst show continuity at all points i' ^ 0. 

Let uq > 0, and let e > 0 be given. Choose a h > 0 such that for all v G {uq — 5, lyo + 6), 

iir-r°iioo<6. 


This implies 


I sup/''(6») - sup/"'°(6')| < e ^ \i{u) - £{uo)\ < e, (129) 

e e 

which establishes continuity of i at all points vq > 0. 

Turning towards the v = 0 case, we define 


0*{u) = argmax/'^(0). (130) 

0 

Proving the continuity of £ at = 0 is slightly more challenging than the corresponding proof in 
Theorem 7.1 from Section 7, since we do not know 0*(z^) explicitly in terms of v. Despite this, we 
can still prove the following lemma: 

Lemma 8.9 (Proof in Appendix D.8). The following equality holds: 

hmsup0*(i^) = 0. (131) 

Now let t'o = 0 and let e > 0 be given. Using continuity of A*, choose an r/ > 0 such that 


— A*(l — 0) — n((T, p)\ < e/2 for all 9 G [0, rj). 


(132) 


Using Lemma 8.9, choose a (5i such that 


9*{n) < rj for all u G [0, (5i). 


For all n G [0, (5i), we have 


= sup 
e 


/IX 2T:ev 

-A (I - 0) + - log 


w , 27rei/ 

, e 0, 27rez^ 

< v{a, p) + - + sup - log —— 

L Q L u 

(b) e 

< v{a,p) + - + 7ru 


(133) 

(134) 

(135) 

(136) 

(137) 


where (a) follows by inequalities (132) and (133), and (6) follows from an evaluation of the supremum 
in (a). Choose ^2 = ^, and choose 5 = min((5i, 52)- We now have that for all n G [0, (5), 

i{i') < v{a, p) + e. (138) 

This combined with i{u) > i(0) = v{a,p) gives \i{n) — ^(0)| < e, thus establishing continuity at 

i^o = 0. □ 


Using Theorem 8.1, we establish the following asymptotic capacity result: 
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Theorem 8.10. The capacity C of an AWGN channel with {a, p)-power constraints and noise 
power V satisfies the following: 

1. When the noise power n ^ 0, capacity C is given by 

C = v{a, p)-]^ log 2-Kev + e{u), 
where e(-) is a function such that = 0. 


2. When noise power u ^ oo, capacity C is given by 


C 


1 (P 





Proof of Theorem 8.10. Note that all the logarithms used in this proof are taken to be at base e. 
1. Using the lower bound in Theorem 3.2, 


C* > ^ log ( 1 + 


1 


fiv{a,p) 

2TTei' 


= v{a, p) - 2 2vrez/ + log + 
= v{a,p) - ^ log27rez/ + 0(i/). 


2Tiev 

g2v{a,p) 


By continuity of i at 0, we have that as —)• 0 

i{u) = v{a,p) + e(z/) 

for some e(-) satisfying lim,y_ 5 .o e{n) = 0. This gives the upper bound 

C < v(a, p) — - log 2neu + e{n). 


(139) 

(140) 

(141) 

(142) 

(143) 


Our claim follows from the inequalities (141) and (143). Unlike the case of cr = 0, we are 
unable to give any precise rate at which e(i^) goes to 0. Since we don’t know what the intrinsic 
volumes of Sn{o',p) are, we can only say that — A*(l — 9) is continuous at 9 = 0, while not 
knowing how fast it approaches v{a, p) as 9 ^ 0. 


2. Note that C is bounded from below by the capacity of an AWGN channel with an amplitude 
constraint of ^/p. Using Theorem 7.6 we obtain for n —?■ oo. 







In addition, the upper bound from Theorem 3.2 states that 



The claim now follows from equations (144) and (145). 


(144) 


(145) 


□ 
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9 Conclusion 


In this paper, we studied in detail an AWGN channel with a power constraint motivated by energy 
harvesting communication systems, called the (a, /?)-power constraint. Such a power constraint 
induces an infinite memory in the channel. In general, finding capacity expressions for channels 
with memory is hard, even if we allow for n-letter capacity expressions. However, in this particular 
case, we are able to exploit the following geometric properties of {Sn{(T,p)}- 

A- . Sm+n (a,p) C Sn{cr,p) X Sm{cr,p), 

B : [Sm{cr,p) X Ofc] x [Sn{cr,p) X Ofc] C Sm+n+ 2 k{cr, p), when k= 

Property (A) allowed us to upper-bound channel capacity, and property (B) allowed us to lower- 
bound the same. In Section 2, we used these two properties to establish an n-letter capacity 
expression. 

The main contribution of Section 3 was the EPI based lower bound. To arrive at this lower 
bound, we used the n-letter capacity expression from Section 2, and the following property: 

C : The limit lim^ ^ log Vol(5n(cr, p)) exists, and is finite. 

For most reasonable power constraints, an exponential volume growth rate as defined in property C 
can be shown to exist. The case of {a, /9)-constraints was especially interesting, because it was fairly 
easy to evaluate v{a, p) using the numerical method in Section 5. We attribute this ease to the 
existence of a state an, which is a single parameter that encapsulates all the relevant information 
about the history of the sequence. We used the computed value of v{a,p) to plot the EPI based 
lower bound. Our results show that energy harvesting communication systems have significant 
capacity gains even for a small battery. We then established an upper bound on capacity using 
the exponential growth rate of volume of the Minkowski sum of Sn{a, p) and a ball of radius 
For the special case of cr = 0, which is the peak power constrained AWGN channel, we explicitly 
evaluated this upper bound. This enabled us to derive new asymptotic capacity results for such a 
channel. We also established a new upper bound on the entropy h{X -|- Z), when X is amplitude- 
constrained, and Z is variance-constrained. The analysis for the case of cr > 0 was more involved 
because the intrinsic volumes of Sn{a,p) are not known in a closed form. Using a new notion of 
sub-convolutive sequences, we showed that the logarithms of the intrinsic volumes of {Sn{a,p)} 
when appropriately normalized, converge to a limit function. We then established an asymptotic 
capacity result in terms of this limit function. Our analysis crucially depended on both, property 
(A) and property (B). It would be interesting to study how our methods can be adapted to study 
power-constrained channels when the constraint does not satisfy one (or both) of the properties 
(A) and property (B), and we intend to pursue this in the future. 


Appendices 

A Proofs for Section 4 

A.l Proof of Lemma 4.2 

If we scale both a and p by some a > 0, by equation (4), Sn{aa,ap) is a \/a-scaled version of 
Sn{a,p). This means that Vn{aa,ap) = a'^Vn{a, p); i.e., v{aa,ap) = log^/a + v{a, p), which proves 
the lemma. 
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A.2 Proof of Lemma 4.3 


Let a G ( 0 , 00 ). Let e > 0 be given. We will show that there exists a 5* > 0 such that for all 
a' G (fj — (5*,cj + (5*), 

|ni(fT') - t>i(cj)| < e. 

Since vi is a non-decreasing function, it will be enough to show that 

ni((T -|- 5*) — — (5*) < e. 

Pick any 0 < 5 < max((T, For 2 ; G (0,1), let 5n((T -|- (5,1) x denote the set Snio + S, 1) 

scaled by \/l — z. Fix z = 25. We will now show that Sn(cr -|- 5,1) x \/l — z C Sn (a - S, 1). 

Any (xi, X 2 , • • •, x„) G Sn(o' + 5,1) satisfies 

i 

x^ <(/ — A;) + cr -|- 5 for all 0 < /c </< re. (146) 

i=k-\-l 

Let the (xi,..., x^) be (xi,..., Xn) scaled by \/l — z. If (xi,..., Xn) happens to lie in Snicr — 5, 1), 
then so does the scaled version (xi,..., x^). If (xi,..., Xn) G 5n(cr + 5,1) \ 5 „((t — 5,1), then for 
each choice of 0 < A: < / < re such that 

i 

{I — k) + a + 6 > ^ xf > {I — k) + a — 5, (147) 

i=k-\-l 

the point (xi, • • • , Xn) satisfies 

xi= xf-z Y, (148) 

i=k-\-l i=k+l i=k-\-l 

< [{I — k) + a + 6] — z[{l — A:) + fj — 5] (149) 

(a) 

< [{I — k) + a + 6] — z (150) 

= {I — k) + a — S, (151) 

where (a) follows since ^ — A: > 1 and a — 6 > 0, implying that (/ — A:) + cj — 5> 1, and (b) follows 
by the choice 2 ; = 25. Thus, the point (xi,..., Xn) lies in the set Sn^cr — 5,1). The containment 

\/l-25 X Sn{(T + 5,1) C Sn{a - 5,1) C 5„(fT + 5,1) (152) 

gives 

^ log(l - 25) -I- xi(cr + 5) < xi(cT - 5) < xi(cr -I- 5). (153) 

Hence, we have 

xi(cj + 5) - xi(fT - 5) < log(l - 25). (154) 

Picking 5* small enough to satisfy 

-^log(l - 25*) < e, 

we establish continuity of xi(cj) in the open set (0, 00 ). 
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Now consider the case when a = 0. We will show that there exists a <5* > 0 such that for all 
a' G [0,r), 

\vi{a') - i;i(0)| < e. 

Since vi is a non-decreasing function, it will be enough to show that 

ui((5*) — ui(0) < e. 

Pick any (5 < 1. Using the same strategy as before, we can show that Sn{5, 1) x y/l — 6 C 5„(0,1). 
This gives 

(155) 


and thus 




0 < ui(5) - ui(0) < log(l - (5). 


(156) 


Choosing 6* small enough such that — ^ log(l — 6) < e, we establish continuity at u = 0. 

A.3 Proof of Lemma 4.4 

For every n, define the function Vn{cr) = We’ll first show that Vn{a) is concave. 

Dehne the set ^n+i U as follows: 


^n+l — { (^1 ) • • • ) ^nt ^x) \ (^1 ) • • • ) ^ n ) £ SnicTxt f )}■ 


(157) 


We claim that S^n+i is convex. Let x = (xi,..., Xn,(Tx) and y = (yi,..., i/n, (Jy) be in ^n+i- For 
A G [0,1], consider the point Ax + (1 — A)y. For any 0 < k < I < n, we have 

i III 

{Xxi + (1 - X)yif = ^ xf + (1 - A)^ Y yf + 2A(1 - A) ^ XiVi (158) 

i=k-\-l i=k-\-l i=k-\-l i=k-\-l 

I I I 

< A^ Y^ + (1 “ A)^ Y^ yf + A(1 “ A) ^ {xf + yf) (159) 

i=k-\-l i=k-\-l i=k-\-l 

I I 

= A X? + (1 - A) yf (160) 

i=k-\-l i=k-\-l 

< (Xax + (1 - A)cry) + (/ - k). (161) 

Thus, Ax -|- (1 — A)y G S^n+i-, which proves that ^n+i is a convex set. 

Now the n-dimensional volume of the intersection of XX^n+i with the hyperplane ax = cr 'is simply 
the volume of Sn{a, 1). Using the Brunn-Minkowski inequality [12], we see that Vol(5n((T, 1))" is 
concave in u, so the logarithm is also concave. This establishes the concavity of Vn(a). 

To show that ui(cj) is concave, we simply note that it is the pointwise limit of the sequence of 
concave functions { 14 }- 


A.4 Proof of Lemma 4.5 

For x" G An{a{n)), the state at time n is nonnegative. Suppose that after time n, we impose a 
restriction that the power used per symbol cannot be more than This means that the battery 
will charge by at least ^ at each timestep, and after 2a{n) steps, the battery will be fully charged to 
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a{n). Denote the set of all such (n + 2cr(n))-length sequences obtained by this process as An{(y{n)). 
This set is contained in Sn+ 2 a{n)AA)^ I); its volume is 

VolMn(cT(n))) X 

The key point is to note the containment 

An{(T{n)) X • • • X AnAin)) C Sm(n+ 2 a{n)){cr{n), 1) , 

for all m > 1, where there are m copies in the product on the left hand side. This holds because 
we ensure that the battery is fully charged to (T{n) after each (n + 2(T(n))-length block. Taking the 
limit in m and using Lemma 3.1, we see that 

vi{a{n)) > ^ \a{n) x • 

Letting n tend to infinity and using conditions (48a) and (48b), we arrive at 

liminf tii(iT(n)) > -log27re, 

n^oo 2 

which proves the claim. 

A.5 Proof of Lemma 4.6 

The key to proving Lemma 4.6 is to examine the distribution of the burstiness a{XA, when is 
drawn from a uniform distribution on An- Since a high-dimensional Gaussian closely approximates 
the uniform distribution on An, it makes sense to look at the burstiness of X” when each Xi is 
drawn independently from a standard normal distribution. 

Let Xi, X 2 ,..., Xn be i.i.d. standard normal random variables. Let T) = Xf — 1, for 1 < i < n. 
These Yi are i.i.d. with zero mean and variance 2. Define S'o = 0 and 

m 

Sm = Yi, for 1 < m < n. 
i=l 

Dehne the burstiness of the sequence of Xj, by 

= max Yfc+i Yk +2 + ■■■ + Yi= max {Si - Sk)- 

0<k<l<n 0<k<l<n 

The following inequality holds: 

Sji < max Si — min Sk ■= (162) 

0<l<n 0<k<n 


Fix some a > 0. Then 

liminfP(E„ <aAn) > hminfP(I]„ < a^n) 

n n 

r ■ Si . Sk ^ a 

= lim mi F[meiX , — mm —^ < 




\/2n \/2 


*■= P{ max Bt — min Bt < -^) 
o<t<i o<r<i A2 

> P{ sup \Bt\ < 

o<t<i 2v2 


( 163 ) 
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where in equation (163), Bt is the standard Brownian motion and the equality in step (a) follows 
from Donsker’s theorem [20]. We now choose a large enough so that 


Pi sup \Bt\ < > j. (164) 

o<t<i 2 v 2 4 

Since limn, P{X'^ G An) = 1/2 by the central limit theorem for 11 , 12 ; • • ■, we have 
liminf P(.A„(a-v/n)) = liminfP(Sn < a^/n,X'^ G An) > 7 , 

n n 4 

where PiAnic(^/n)) := PiX"^ G Anio^^/n)). The volume of Anio^^/n)) is upper-bounded by the 
volume of An- Furthermore, it is lower-bounded by the volume of a ball Bn centered at the 
origin, such that P(X"' G Bn) ='■ P{Bn) = P{Anioiy/n)), since a Gaussian distribution decays 
radially. Using standard concentration bounds on the normal distribution [ 21 ], to satisfy P{Bn) > 
1/4, the radius of Bn must be y/n -|- oiy/n). Thus, lim^ _ 4log27re. Using Yol{Bn) < 

\o\{An{(y-\/n)) < Vol(^n), we obtain lim„ _ liog27re. Thus, with cT(n) = ay/n, the 

pair Anic^in)) and cT(n) satisfy both the conditions in Lemma 4.5, thereby proving Lemma 4.6. 


B Appendix for Section 5 


Let 0 < 7 < 1. We define a new set 5^,7 (u, 1) to be 


Sn,-yio', 1) = {x” G Snic, p) \ xj > ^ for every 1 < i < n}. 
Using Fekete’s Lemma, it is easy to establish that following limit exists: 

Vol(5„,7(cr, 1)) 


lim 

n^oo 


n 


:= Ui,7(cr). 


(165) 


(166) 


Clearly, vi^^ia) < ui((t) as 5 , 1 ^ 7 (a, 1) C 5n((T, 1). In Lemma B.l, we show that it is possible to 
choose a small enough value of 7 such that vi^-y{a) approximates viia) as closely as desired. 

Lemma B.l. IFe have 


vi 


\ -h)< Vi,^i(^) < 'yi(o-). 


where rj = 'y + 2{y/a + ^y/Y)- 

Proof. Clearly, ui^ 7 (cr) < viia), since 5 n, 7 ((T, 1) C 5 n(( 7 ,1). 

Now let G 5„(fT, 1 — ry) n M". We claim that 

(a:i + VT; • ■ ■, Xn + V7) e Sn,^{cr, 1). 

This would imply that a translated version of 5n((T, 1 — ? 7 )n]R” lies inside 5 ^, 7 (cr, l)nM!/, which will 
give us a lower bound on the volume of the latter in terms of the former. Since each (xj -|- ^/y)^ > 7 , 
the only condition we need to check is whether (xi -|- ^/y,..., x^ + y/Y) £ ‘^^(cr, 1). For any 
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0 < A: < / < n, we have 
i 


+ V7)^ = x‘f + 2y/^ J2 Xi + {1- k)j 


i=k+l 


i=k+l 


i=k-\-l 


This gives us 


implying that 


< {I - k){l - r]) + a + 2^ ^ Xi + {1- k)-^ 

i=k-\-l 

I 

< {I - k){l - r] + j) + a+ 2^y^ ^ Vcr +l 

i=k-\-l 

< {I - k){l - r] + j + 2^^/a + 1) + a. 

= (J, — k) + a 


Vol(5n(cr, !-??)< Vol(5n,7(cr, 1)) < Vol(5n(cr, 1)), 


(f^) + \ ^ M(^)- 


(167) 

(168) 

(169) 

(170) 

(171) 


(172) 

(173) 


(174) 


By the continuity of ui((t), we see that choosing a 7 (and consequently an r]) small enough will give 
a value of that is as close as desired to t'i(<T). □ 

Lemma B.l ensures that a numerical method which can closely approximate can also 

be used to closely approximate vi{a) for small values of 7 . Henceforth, we focus our attention on 
calculating vi^^{a). As noted in Section 5, we exploit the idea of battery state. Given (xi,..., Xn) G 
Sn,'y{cr, 1 ), define 

fan if (Tn< O', 

'Pn — \ 2 -n 

fT„_i + l-xt if an = a. 


(175) 


Setting (pQ = a, equation (175) can also be written as 


+ 1 - xl if (pn-l < O', 


a + 1 — xz 


if > O'. 


(176) 


Define the function <1>„ : Sn,'-/{a, 1) —)• M such that ^n{xi, ■ ■ ■, Xn) = </>n- Let A„ be the Lebesgue 
measure restricted to Sn,'y{a, 1). Let i^n be the measure induced by 4>„ on M. In Lemma B.2 below, 
we show the following: 

Lemma B.2. The measure Vn is absolutely continuous with respect to the Lebesgue measure on M. 
Proof. We first calculate Define 
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We have the relation (pi = a+l—xf, where xi has the Lebesgue measure on 1): [—^/(t + 1, —\/ 7 ]U 

[y/a + 1, . It is easy to see that 


Flip) 


0 for (/) < 0 

< 2ii/a + 1 — \Ja + 1 — p) forO<(/><fT + l — 7 
2{\/a + 1 - for fj + 1 - 7 < 


(177) 


Observe that Fi, being Lipshitz, is an absolutely continuous function. This implies that the measure 
vi is absolutely continuous with respect to the Lebesgue measure on M and possesses a Radon- 
Nikodym derivative /i, which equals the derivative of Fi almost everywhere. We set /i as follows: 


hiP) 


'0 

< 1 
^(T+I — Cf) 

lo 


for (/) < 0 

for 0 <(/)<(T + l —7 
for cr + 1 — 7 < (/). 


(178) 


We note that /i is continuous and bounded on the closed interval [0,(T + 1 — 7 ]. Our proof now 
proceeds by induction. We assume that the measure Un admits a density /„, which is continuous 
and bounded on the closed interval [ 0 , u + 1 — 7 ], and prove that Vn+i has a density fn+i which is 
continuous and bounded on [ 0 , cr + 1 — 7 ]. 

Define 

Fn+lip)) = r'n+l((-00, (/)]). 


Since Vn is supported on [0,(T + 1 — 7 ], we can use the expression in (176) to conclude the same 
about t'n+i and restrict our attention toO<0<(T + l — 7 . For p in this range, we use relation 
(176) and express F^+i in terms of /„ as follows: 


_i+T, 


^(p-i+7 I- 

P+liP) = / / 

J a;=0 J t- 

rcF pa 

Jx=d}-\-l—'y Jt= 


_ dtdx 

a;=0 Jt='y yt 

fnix) 


+ I j "' dtdx 


+ 


I x=ij Jt=a—{(l)—l) ^/i 

r*(/)— 1+7 


-dtdx 


r(p—L-\-'y 

/ 2 fnix)[Vx + 1 - y/^dx 

Jx=0 


+ 2 fnix)[Vx + l- i/x-ip- l)]dx 

J x=(l)—l-\-'y 

ra+l-'Y _ _ 

+ / 2fnix)[i/(T + I - s/a - ip- l)]dx. 


(179) 


(180) 


From the induction assumption of continuity and boundedness of /„, it is easy to check that Fn+i 
is Lipshitz and therefore absolutely continuous. This implies that r'n+i permits a density, which is 
equal to the derivative of Fn almost everywhere. We can evaluate this density by differentiating 
i7_|_i with respect to p. This involves differentiating under the integral sign, and the conditions for 
doing so are seen to be satisfied because of the continuity and boundedness of fn and the square 
root function. We then get 


r~'' / 

J (P-l+'y yx - [p - l) Ja ya-[p-l) 

which is supported on, and is bounded and continuous on, the interval [ 0 , c + 1 — 7 ]. □ 
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Equation (181) in the proof of Lemma B.2 describes the evolution of fn as the dimension n 
increases. Let C'([0, u +1 — 7 ] be the set of continuous functions defined on the interval [0, u +1 — 7 ]. 
Define the integral operator ^ : (^([O, cr + 1 — 7 ] —>• (^([O, u + 1 — 7 ] as follows: 


A{x, t) 


^/x+l—t 

1 

\/a+l—t 

0 


ifO<x<(T, 0 <t<x + l — 7 , 
ifcr<x<(T + l— 7 , 0 <f<cr + l — 7 , 
otherwise. 


(182) 


We can express equation (181) in another form, 


fn+l{t) = j A{x,t)fnix)dx. 

We denote this fn+i = A{fn). Iterating this relation, we obtain 

fn+l=A^f,. 


(183) 


(184) 


We make three crucial observations. Firstly, the kernel A is bonnded and piecewise continuous with 
the discontinuities confined to a single curve t = x + l — 7 . It is also immediate that the spectral 
radius of A, defined by 

r{A)= sup \\Af\\ 

11 / 11=1 


is such that r(A) > 0. We use Theorem 2.13 from Anselone [22] to obtain that such an operator A 
is compact. In addition, we can apply the Krein Rutman theorem from Schaefer [23] to establish 
that r{A) is an eigenvalue with a positive eigenvector u G ^/([O, u + 1 — 7 ] \ 0. 

Secondly, we have 

rcr+l —7 

Z/ndOjO-T 1 - 7 ]) = / fn{x)dx = Yol{Sn,-y{(J,l). 

Jx=0 


Thus we have 


111 , 7 ( 0 ')= lim - logVol(5„,.^(cr, 1)) 

(185) 

n—>-oo n 

1 

pcr+l—’y 

= lim — log J 

1 fn{x)dx 

(186) 

n—>-oo n J 

x=0 

= lim — log J 

pcr+l—’y 


1 A^~^fi{x)dx 

(187) 

n—>-oo n J 

x=0 

r{A) 


(188) 


where (a) follows because the projection of fi in the direction of u is nonzero owing to the positivity 
of both these functions. 

Thirdly, define a sequence of operators {An} as discrete approximations of A as follows. Let 


^nf{t) = '^A{jhn,t)f{jhn)hn. 

i=o 

Using Theorem 2.13 from Anselone [22] once more, we conclude that the sequence of operators {An} 
is collectively compact and that IJAnj] —)• [[Ajj. We can now use existing numerical techniques to 
find r{An), which will provide an approximation to r{A). The spectral radius r{A) equals ni^.y(cr), 
which closely approximates vi{a), and validates the numerical procedure as described in Section 5. 
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C Proofs for Section 7 


C.l Proof of Lemma 7.2 


Denote = [—A, A]”, and = Bn{\/nv). Let Cn = An © Bn- Note that for any m,n> 1 

Bn{Vm) X Bm{y/mv) C Bm+ni^/{m + n)v) (189) 

[-A, A]^ X [-A, A]™ = Ar+^. (190) 

It follows that 


Cm X Cn — {Am © Bm) X {An © Bn) (f91) 

= {Am X An) © {Bm X Bn) (192) 

© Bm+n (193) 

= Cm+n- (194) 


This implies 


Vol(C^+n) > Vol(C',,)Vol(C^), (195) 

which immediately implies existence of the limit lim„_^oo ^ log Vol(C'„), which equals £{ 1 ^) as de¬ 
fined in equation (71). To show this limit is finite, we note that An © Bn{'\/ nA?). Thus 
Cn © Bn{\/n{y/u + ^)), wluch gives 


£{^) < 2 loS 27re(\/7 + A)‘^ < 00 . 


C.2 Proof of Lemma 7.3 

We have the trivial bounds 


Yo\{[-A,Af®Bn{V^)) 

n + 1 


< en/-(e„) < Vol([-^,^]” © Bn{V^)), 


(196) 


which implies 

ilogVol([-^,A]''©B„(\/77)) - </^(0„)< ^\og\o\{[-A,A]^ ® Bn{\/m)). (197) 

Taking the limit in n and using Lemma 7.2 we see that 

lim /n(i9n) = f(z^). (198) 


C.3 Proof of Lemma 7.4 

We first prove pointwise convergence. Looking at equation (78), we see that all we need to prove 
is that for all 6 G [0,1], 


lim — log 

n^oo Ti 


r(n + l)n"^/2 


r(n(l -e) + l)T{ne + l)T{ne /2 + 1 ) 


= 77(0) + ^ log 2e-^ log 0. (199) 


For 0 = 0, we can easily check the validity of this statement. Let 0 > 0. We use the approxi¬ 
mation 

logr(^) = zlogz - z + log -—h o{z). 

ZTT 
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{nO + 1 ) log 


nO + 1 


1 / r(n + l)n”®/2 \ _ 

n \r{n{l -e) + l)T{ne + l)T{ne/2 + 1) j ~ 

if, n + 1 nO n(l — 6) + l 

— ( (n + 1) log- 1 -log n — (n(l — 0) + 1) log — — 

n\ e 2 e 

— {n9/2 + 1) log ^ _|_ o(n)^ . 

Using (x + 1) log(x + 1) = xlogx + o(x), we can simplify the above to get 
1 / TlO - - 

— ( nlogn H-logn — nOlognO — nOlognO — ind 12') logn 0 / 2 e + o(n) 

n \ 2 

= — (nH(6) — (n9j2) log(6*/2e) + o(n)) . 
n 


e 

( 200 ) 


( 201 ) 

( 202 ) 


Taking the limit as n —)> oo, we establish equality (199). 

To show uniform convergence, we first observe that the functions /^(•) are concave. This 
concavity is immediately evident from the log-convexity of the T function and from equation (78). 
Therefore, {/^} are concave functions converging pointwise to a continuous functions f'^ on [0,1]. 
Uniform convergence now follows from Lemma E.l. 


C.4 Proof of Lemma 7.5 

By Lemma 7.4, the sequence of functions {/)(} converges to uniformly. This uniform convergence 
implies that the family of functions {/^} is equicontinuous [24] (Section 10.1, Theorem 3, pg. 209). 
Let e > 0. Choose N large such that |/^(x) — /^(y)| < e/2 if \x — y\ < 1/N. This implies that for 
all n > N, 

max (9) > (9n) > max (9) - e/2. (203) 

u u 

Using the uniform convergence of {/)(}, we choose M large enough such that Wf' — f^Woo < e/2 
for all n > M. Let L = max(M, N). For all n > L, we have 

max/''( 0 ) + e /2 > max/)(( 6 ») > fn{9n) > max/^( 0 ) - e /2 > max/''( 6 ») - e, 

9 9 9 9 

and thus 

l/n(^n) -max/‘^( 6 »)| < e. 

9 

This concludes the proof of equation (82). By Lemma 7.3, we immediately have the equality (83). 

C.5 Proof of Theorem 7.7 

Let e > 0. Let and be n i.i.d copies of X and Z respectively. Let 5n be given by 

6n := p ^ [-^, Ar © BniVniu + e) )) (204) 

Denote 

Cn ■= [—A, Al\^ © Bn{\/n{v + e) ). 

By the law of large numbers, the probability 5n 0. 
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Let Y := X + Z. We have 


(205) 

(206) 


nh{Y) = h{Y^) = H{6n) + (1 - 6n)h{Y^\Y^ G Cn) + Snh{Y^\Y^ i C^) 
< E{bn) + (1 - <5n) log Vol(a) + 6nh{Y^\Y^ ^ Cn). 

Let Y"^ ~ p{Y^\Y'^ ^ Cn). We have following bound on Y"' 

L;[||y”||2] < n{v + A^). 


This translates to a bound on Y^ 


E[||y”|p| < 


which implies 


Substituting in inequality (206), 


HY') < \ log 

2 On 


which implies 


h{Y^) < H{6n) + (1 - dn) log Vol(a) + <5^ log + 

2 On 


h{Y) < ILW + (1 _ ^__)iogvoi^ ^ 4. i„gg 5T + L ), 

n n 2 6n 


Taking the limit in n, we get 


(207) 

(208) 

(209) 

( 210 ) 

( 211 ) 


h{Y) < e{u + e). 


( 212 ) 


As this holds for any choice of e, we let e tend to 0 and use the continuity from Theorem 7.1 to 
arrive at 


h{Y) < e{u). 


(213) 


D Proofs for Section 8 

D.l Proof of Lemma 8.2 

Let x^,y'^ G Sn{o',p) and let = Ax"' + (1 — A)y". By Jensen’s inequality we have for every 
1 < i < n, 

zf < Xxf + (1 - X)yf. 

Since both x" and y" both satisfy (4), the above inequality gives us that z" does so too; i.e., 
z" G Sn{(J,p). 
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D.2 Proof of Lemma 8.3 

The sets {5 ji(cj, p)} satisfy the containment 

<Sm+n T Sm X Sn for every m,n > 1. (214) 

This implies that the family of intrinsic volumes {fJ-n{')}n>i, is sub-convolutive; i.e., it satisfies the 
following condition: 

fJ'm+n for every m, n > 1. (215) 

Noting that Unin) is the volume of Snio',p), and /in(0) = 1 for all Sn, we can check that the 
sequence {pn(')} satisfies the assumptions (A), (B) and (C) detailed in Appendix F; namely, 

(A) : a := lim —logfin{n) is finite. 

n^oo 77, 

(B) : /3 := lim — logpn(O) is finite. 

n^oo 77 

(C) : For all n, Pnin) > 0,/in(0) > 0. 

Lemma 8.3 then follows from the results in Appendix F, in particular Lemma F.l. 


D.3 Proof of Lemma 8.4 

Note that the claims in points 1 and 2 immediately imply 3, since = an+ b'^. 

We shall prove 2 first. The expression for h^{9) is given by 

Since the Gamma function is log-convex [25] (Exercise 3.52), we see that 65)(-) is a concave function. 
To show 1, note that all we need to prove is that 

for all 1 < j < n - 1, (217) 

as an is a linear interpolation of the values at A This is equivalent to proving 

Pn(j)^ > Pn(j - l)Pn(j + 1) for all 1 < j < u - 1. (218) 

This is an easy application of the Alexandrov-Fenchel inequalities for mixed volumes. For a proof 
we refer to McMullen [26], where in fact the author obtains 

PnUf > - l)Pn{j + !)• 

3 


^1> 
n 


) + <^r. 


D.4 Proof of Lemma 8.5 

As noted in Appendix D.2, the family of intrinsic volumes {pn{-)}n>i, is sub-convolutive and it 
satisfies the assumptions (A), (B), and (C) detailed in Appendix F. Part 1 of Lemma 8.5 is now 
an immediate consequence of Theorem F.4. 

To prove part 2, let E C M be an open set. We assume that E n [0,1] is nonempty, since 
the otherwise the result is trivial. We will construct a new sequence of functions {pn} such that 
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/^n > An for all n; i.e., fin pointwise dominates jin for all n. The large deviations lower bound for 
the sequence {An} will then serve as a large deviations lower bound for the sequence {fin}- 
For notational convenience, we write for Sn{(y,p) in this proof. Fix an a > 1. Let 7 = 

Let 

5,+., = G G Sa,xlXl = 0}. 

For all A: > 0, the intrinsic volume of a convex body is independent of the ambient dimension 

[18]. Thus, for 0 < A < a, the intrinsic volume of is exactly the same as that of Sa- For 
a + l<A:<a + 7 , the intrinsic volume of 5^+.^ equals 0. The sequence of intrinsic volumes of 
Sa+'y may therefore be considered to be simply fia- In addition, note that for all m > 1, 

'Sa+'y X • • • X Sa-\--y ^ 'Sm(a+'y)j 


which implies 


ka X • • • X fla ^ km{a+y) 


This leads us to define the new sequence An as 


* 1^1 

An — ka '*'•••* fla ■— fJ-a 

' -V-' 

I I 

L (1+7 J 


Clearly fin < fJ-n- Define Gn{t) as follows. 


Gnit) = log^An(j>-^*, 

j=0 

and consider the limit 

1 '' 1 77 

lim -Gnit) = lim -[—^—JG'a(t) 
n^oo n n^oo n (2 + 7 

^ Gait) 

a + 7 ' 

^ G ft) 

Applying the Gartner-Ellis theorem, stated in Theorem F. 2 , for {An} and noting that is 

differentiable, we get the lower bound 

liminf-logAn/n(^) > - inf (^x), 

n->-oo n x&F \a +'yJ 


(219) 

( 220 ) 


which implies 

T - rli -r « * + l 

limmf -logfin/niF) > - mf —^- x 

n^oo n ' x&F a + 7 \ a 

We claim that inix^F yf^da converges to infa;gi 7 ’A*(x). Let e > 0. We can rewrite the 

infimum as 



inf - 

xeF a + 7 


9a 


0 + 7 


X = 


inf 

y^9^F 0+7 


9*aiy)- 
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Using Theorem F. 6 , we know that {g^} converges uniformly A* over [0,1]. By the converse of the 
Arzela-Ascoli theorem, we have that g'* are uniformly bounded and equicontinuous. Let <5 > 0 be 
such that 


|A*(x) — A*(?/)| < e/3 whenever \x — y\ < 6. 

Let M he a uniform bound on | 5 *(-)|. Choose Aq such that for all a> Aq, 


Choose Ai such that for all a> Ai 


Choose A 2 such that for all a > A 2 , 


7 


a + 7 


M < e/3. 


||5:-A*||oo <e/3. 


Choose A 3 such that for all a > A 3 , 


a + 7 


< <5. 


a + 7 


F n [0,1] / cf). 


Now for all a > max(Ao, Ai, A 2 , A 3 ), 


inf 


, —ffa(y) - inf A*(y) 

^££± 7^0 + 7 yeF 


< 


+ 


inf -—9*a{y) - inf gliv) 


inf 9*a{y)- inf A*{y) 


inf A*(y) — inf A*{y) 

ylzB+Xp yeF 

^ n. 


+ 

(a) 

7 e/3 + c/3 T e/3 
= e. 


( 221 ) 

( 222 ) 

(223) 

(224) 

(225) 


By the relation (225), all the infimums involved in the above sequence of inequalities are finite. 
In step (a), the first term is less that e/3 by inequality (222), the second term is less that e/3 by 
inequality (223), and the last term is less that e/3 by inequality (224). This completes the proof of 
part 2 of Lemma 8.5, and thus completes the proof of Lemma 8.5. 


D.5 Proof of Lemma 8.6 


Note that the claims in points 1 and 2 immediately imply 3, since 

We’ll first prove the claim in point 2. We start by proving pointwise convergence of {^n(-)}- 
Recall the expression for b'^{9), 


1 7r^®/2 

n T[n9/2 + 1 ) 
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For 0 = 0, this convergence is obvious. Let 0 > 0. We use the approximation 


logr( 2 :) = zlogz — z + 0{\ogz)^ 


and get that 


KinO) = - 

_ 1 

n 


n6 nO nO 

— log Tinv - — log — + C)(log n0) 
2 2 2e 


nO ^ 27:ev 

ylog—^ + O(logn0) 


0, 27iev OilognO) 

= -log—— + ^ ^ ^ 

2 0 n 


( 226 ) 

(227) 

(228) 


Taking the limit as n —>■ oo, the pointwise convergence of follows. Concavity of from point 2 
of Lemma 8.4, combined with Lemma E.l then implies uniform convergence. 


We shall now prove point 1. We start by showing the pointwise convergence of a„(0) to — A*(l — 
0), or equivalently the convergence of an(l — 0) to —A*(0). Note that convergence at the boundary 
points is already known. Let 6q G (0,1). For ease of notation, we denote 

X(0) := -A*(0) 

an(0) := a„(l - 0). 

Note that is linearly interpolated from its values at j/n, where an{j/n) = ^ log Unij)- Let e > 0 
be given. The function x, being continuous on the bounded interval [0,1], is uniformly continuous. 
Choose 5 > 0 such that 

\x{x) — x{y)\ < whenever \x — y\ < 6. 


Choose A^o > l/(<^/3), and divide the interval [0,1] into the the Nq intervals Ij := 


for 


0 < j < Nq — 1. Note that each interval has length less than 5/3. Without loss of generality, let 6q 
lie in the interior of the A:-th interval (we can always choose a different value of Nq to make sure 9q 
does not lie on the boundary of any interval). Thus, 

k-1 k 

iVo ^ ° ^ Nq- 


Lemma 8.5 along with the continuity of x imply that 

1 


lim -log//n(7i) = supx(6')- 


n^oo 77 , 


eeii 


(229) 


For n > 2/ min ( 6*0 - tvT ’ No “ 


there exists an i such that 

k — 1 i „ i + 1 k 

< - < 00 < - < —■ 

I\Q n n 1\Q 


Thus for some A > 0, we can write 


an{9o) = A-log^„/„(i/n) + (1 - A)-log/i„/„((i + l)/n) 
n ' n ' 


(230) 


( 231 ) 
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and obtain the inequality 


an{0o) = A- log^„/„(i/n) + (1 - A)- log^„/„((z + l)/n) 

n ' nrt ' 


n 


n 




< - log Ain/n (4). 

n ' 


Thus we have the upper bound 


limsupan(6'o) < lim -logAi„/„(4) 
n n->-oo n 

= sup x(6') 

eeh 

(a) 

< x{0o) + e 


(232) 

(233) 

(234) 

(235) 

(236) 

(237) 


where (a) follows from the choice of A^o and uniform continuity of y. 
Define 


As 


9n{j) = arg sup I - ) . 


i s.t. ie/i 

n n -J 


n 


n 




it is easy to see that 


lim -logAin/n('9n(j)) = 1™ -logAin/n(4) 

n —n ' n —i.rv-i r} ' 


n^oo Tl 


n^oc Tl 

= supx(6')- 

eeii 


Note that 


supan(6') > -logHn/niOnU))- 

eel, n 

This implies that for the intervals 4-i and 4+i) 


lim inf 
n^oo 


lim inf 

n^oo 


sup UniO) 
S&Ik-i 


sup an{9) 
S&Ik+i 


> sup x(<9) > x{0o) - e 

S&Ik-i 

> sup x{&) > xiOo) - e- 


Since an{0) is concave, this implies 


an{9o)>mm{ sup a„(6'), sup a„(6')). 

eeik+i 


Taking the lim inf on both sides, 


lim inf an{6o) > xi^o) - e- 

n^oo 


(238) 

(239) 


(240) 

(241) 

(242) 

(243) 


Inequalities (235) and (243) prove the pointwise convergence of an{0o) to x(^o)- Concavity of a„ 
from point 2 of Lemma 8.4, combined with Lemma E.l then implies uniform convergence. 
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D.6 Proof of Lemma 8.7 


By Lemma 8 . 6 , the sequence of functions {/^} converges to uniformly. Using the converse of 
the Arzela-Ascoli theorem, this implies that the family of functions {/^} is equicontinuous. Let 
e > 0 be given. Choose N large such that |/^(x) — /^(y)| < e/2 if \x — y\ < 1/N. This implies that 
for all n > N, 

max fn (0) > fn 0n) > max (9) - e/2. (244) 

u u 

Using the uniform convergence of {/^}, we choose M large enough such that ||/'^ —/^||oo < e/2 
for all n > M. Let L = max(M, N). For all n > L, we have 

maxf’^ie) + e/2 > max/^( 6 ») > fn0n) > max/^( 6 ») - e/2 > maxf^{e) - e, 

u u u u 

and thus 

\f^{6n) -ma^f "{9)1 < e. 

6 

This concludes the proof. 


D.7 Proof of Lemma 8.8 


Recall that 

Vo1(5„(ct, p)” © Bn{\/nv)) 


We have the trivial bounds 


j=0 


(.nrM < Vol( 5 „(f 7 ,p) © < (n + 


(245) 


implying 

fn0n) < - logVol(5n(<T,p) © R„(\/w)) < /n(^n)- 

n n 

Taking the limit in n, we obtain 

lim fn{9n) = f(0- 

n^oo 

An application of Lemma 8.7 gives 

f(o = supr(0). 

e 

D.8 Proof of Lemma 8.9 

Recall the expression of f'^{9): 

r(9) = -A*(l-(») + 5log^. 

Suppose limsupj^_,.o 9*{i') = rj > 0. Choose a sequence {t'n} such that 


lim iy„ = 0 

n^oo 


9*{j''n) > ^ for all n > 1 . 


(246) 

(247) 

(248) 


(249) 
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(250) 

(251) 




We have that for all > 0 


Thus, 


e{u) = supr( 0 ) > r(0) = -W(l) = v{a,p). 
e 


v{a,p) < £{un) 

= r"(rK)) 


0*{iyn) 


< sup 
e 


_A*(l-()) + ?log^ 


j_ »•("»), 

H-;;— log i^n 


< C-\ - - —logi/„ 

(b) ri 

< C + ^logUn 


(252) 

(253) 

(254) 

(255) 

(256) 

(257) 


where in (a), C is a constant and in ( 6 ) we assume \ogVn < 0. Taking the limit as n —)• oo, we get 
that 

v{(y,p) < lim C + y logt'n = —oo, (258) 

n^oQ 4 

which is a contradiction. Thus, it must be that limsup^_,.g 0 *(j^) = 0. 


E A convergence result for convex functions 

Lemma E.l. Let {fn} be a sequence of continuous convex functions which converge point wise to 
a continuous function f on an interval [a, h]. Then fn converge to f uniformly. 

Proof. Let e > 0. We’ll show that there exists a large enough N such that for all n > N, | |/n— /| |oo < 

e. 

The function / is continuous on a compact set, and therefore is uniformly continuous. Choose 
a (5 > 0 such that |/(x) — f{y)\ < e/10 for \x — y\ < 5. Let M be such that (6 — a)/M < 5. We 
divide the interval [a, 6] into M intervals, whose endpoints are equidistant. We denote them by 
a = ao < «! < • • • < aM = b. Since fn{c(i) —)• f{ai), there exists a W such that for all n > Ni, 
\fn{ai) - f{ai)\ < e/10. Choose N = max(M,iVo, • • • ,Nm). 

Consider an x G (Q!j,Q;j+i) for some 0 < i < M, and let n> N. Using uniform continuity of /, 
we have 

f{ai) - e/10 < f{x) < f{ai) + e/10. (259) 

Further, we also have 

fnioti) < f{oii) + e /10 , (by pointwise convergence at ai) 
fn{ai+i) < /(cKj+i) + e /10 , (by pointwise convergence at erj+i) 

< f{ai) + 2 e /10 .(by uniform continuity of /) 

Convexity of fn implies 


fn{x) < max(/n(Q!i), fn{ai+i)) < f{ai) + 2e/10. (260) 
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Combining part of equation (259) and equation (260), we obtain 


fn{x) - f{x) < 3e/10. 


( 261 ) 


We’ll now try to upper bound fn{x). First consider the case when i > 1. In this case we have 


Qfj-i < ai < X < Oj+i. 


We write a* as a linear combination of x and cti-i, and use the convexity of fn to arrive at 


/n(w) < 


C^i—1 
X — ai-i 


fn{x) + 


X — ai 
X — ai-i 




This implies 


X 0:2—1 f / \ ^ f \ ^ -p t \ 

~Jn{Oii) Jn\(^i—l} ^ Jn\X). 


O2 1 ai ai—\ 

Taking the infimum of the left side, we get 


inf 


xe{ai,ai+i) ai — ai-i 


X Oj-l f ( \ X Oti . \ ^ f / \ 

Jn\(^i) Jn(Oj—1) < Jn\X)- 




Note that since the LHS is linear in x, the infimum occurs at one of the endpoints of the interval, 
ai or Oj+i- Substituting, we get 


fn{x) > umi{fn{ai),2fn{ai) - fn{ai-i)) 

> min(/(a 2 ) - e/10, 2{f{ai) - e/10) - /(oi-i) - e/10) 

> min(/(Q;i) - e/10, 2f{ai) - /(oj-i) - 3e/10) 

> min(/(a;j) - e/10, 2f{ai) - /(a*) - e/10 - 3e/10) 

= fiai) - 4e/10. (262) 

Combining inequality (262) with a part of inequality (259), we have 

fn{x) - f{x) > -5e/10. (263) 

Combining (261) and (263) we conclude that for all x G (oi, om); and for all n > N, 

\fnix) - fix)\ < e/2. (264) 

Now let X G (cKojOi)- We can establish inequality (261) for x G (oojOi) using the same steps 
as above. We express oi as a linear combination of x and 02 and follows the steps as above to 
establish (263) for x G {ao,ai). This shows that for all x G [a,6], \\fn{x) — /(x)|| < e/2 for all 
n > N, and concludes the proof. □ 


F Convergence properties of sub-convolutive sequences 

Consider a sequence of functions {fJ-n{-)}n>i, such that for every n, fin ■ —)■ M+ with finU) — 0 
for all j > n + 1. We call such a sequence of functions a sub-convolutive sequence if for all m,n > 1 
the convolution fim* fin pointwise dominates fim+n'-, i-e., 

Aim * fJ^nii) > fJ-m+nii) for all i > 0, and for all m, n > 1. (265) 
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For our results on sub-convolutive sequences, we make the following assumptions: 


(A) : a := lim — log/in(u) is finite. 

n^oo 77, 

(B) : /3 := lim — log//n(0) is finite. 

n^oo 77 

(C) : For all n, iJ.n{n) > 0,/in(0) > 0. 

Note that fim * IJ-nipi + n) = ijLn{n) and * /^n(0) = ^ni^)■ Thus, the existence of 
the limits in assumptions (A) and (B) is guaranteed by Fekete’s Lemma, and we have 

a = inf — log Unin) (266) 

n n 

/? = inf - log//„(0). (267) 

n n 

For n > 1, define Gn : M —>■ M as 

n 

Gn{t) = log ^ /in(j>-^*- (268) 

j=0 

Condition (265) implies that the functions Gn satisfy the inequality, 

G'm(t) + Gn{t) > Gm+n{t) for every m, n > 1 and for every t. (269) 

G ft) 

Thus for each t, the sequence {G„(t)} is sub additive, and by Fekete’s lemma the limit lim„ ' 
exists. To simply notation a bit, define gn ■= ^ and let A be defined as the pointwise limit of 
9 ns; i.e., 

A(f) = lim5r„(f). (270) 

n 

Lemma F.l. The function A satisfies the following properties: 

1. For all t, 

max(/3, f + a) < A(f) < gi(f) (271) 

2. A is convex and monotonically increasing. 

3. Let A* he the convex conjugate of A. The domain of A* is [0,1]. 

Proof. 1. The inequality (269) immediately gives that for all t, and all n > 1, 

nGi{t) > Gnit), which implies gi{t) > gn{t)- (272) 

Taking the limit in re, it follows that A{t) < gi{t) for all t. 

For all re, the functions gn are monotonically increasing, and for all t they satisfy 

gnit) > lim gnit) =-log finiO). (273) 

t—^—oo re 

In addition, we also know that 

inf - log^n(O) = (3. 

n n 
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This gives us that 


( 274 ) 


9nit) > 13. 

Taking the limit in re, we conclude that for all t, 

A{t) > /?. (275) 

For all re, we have the lower bound on gn given by 

1 ” 

gn{t) = - log V (276) 

n 

j=0 

> -log/rn(re)e"* (277) 

n 

= t + -\oggn{n) (278) 

re 

(a) 

>t + a (279) 

where (a) follows as 

inf - log/rri(re) = a. 

n re 

Taking the limit in re, we conclude that 

A{t)>t + a. (280) 

Equations (275) and (280) establish 

A{t) > max(/?, t + a). 

2 . The functions {gn} are convex and monotonically increasing. Since A is the pointwise limit 
of these functions, A is also convex and monotonically increasing. 

3. Note that the convex conjugates of the functions gi{t) and max(/3, t + a) are both supported 
on [0,1]. Since A is trapped between these two functions, it is clear that A* is also supported 
on [0,1]. 

□ 

Theorem F.4 requires an application of the Gartner-Ellis theorem [14], which we state here for 
reference: 

Theorem F.2 (Gartner-Ellis theorem). Consider a sequence of random vectors Zn € where 
Zn possess the law gn o.nd the logarithmic moment generating function 

A„(A) := logE[exp{X,Zn)] • 


We assume the following: 

(*); For each A G the logarithmic moment generating function, defined as the limit 

A(A) := lim —A„(reA) 
n—>-00 re 

exists as an extended a real number. Further the origin belongs to the interior T)\ := {A G 
I A(A) < oo}. 
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Let A* be the convex conjugate of X with 2?a* = {x G | A*{x) < 00 }. When assumption {-k) 
holds, the following are satisfied: 

1. For any closed set I, 

1 

lim sup — log//„(/) < — inf A*(a:). 

n^oo x<^I 

2. For any open set F, 

liminf — log/in(-?^) > — inf A*(x), 

n^oo n xSFnJ' 

where T is the set of exposed points of A* whose exposing hyperplane belongs to the interior 
ofVA. 

3. If A is an essentially smooth, lower semicontinuous function, then the large deviations prin¬ 
ciple holds with a good rate function A*. 


Remark F.3. For definitions of exposed points, essentially smooth functions, good rate function, 
and the large deviations principle we refer to Section 2.3 of [14], For our purpose, it is enough to 
know that if A is differentiable on = 1^^; then it is essentially smooth and A* satisfies the large 
deviation principle. 


Theorem F.4. Let {pn}n>i be a sequence of sub-convolutive of functions as defined in equation 
(265), satisfying assumptions (A),(B) and (C). Define a sequence of measures supported on [0,1] 
by 


hn/n := hn{j) for j > 0 . 

Let I CR be a closed set. The family of measures {pn/n} satisfies the large deviation upper bound 


limsup — log/i„/„(/) < — inf A*(x). (281) 

n^oo n xei 


Proof. Let Yhj Pn{j) = Sn- We first normalize Pn/n to define the probability measure 


f^n/n 
Pn := - 


The log moment generating function of pn, which we call Pn, is given by 


Thus, 


Pn{t) = \og'^pn{j/n)e^^/'^ 

i=o 

1 

= log —^/rn(j)e^‘/' 

3=0 

= Gn{t/n) - logSn. 


lim —Pn{nt) 

n^oo 71 


lim (GlW 

n^oo y n 


log Sn \ 
n J 


= A(t)-A(0). 
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Note also that by Lemma F.l, the function A is finite on all of M, and thus 0 lies in the interior 
'D(A). Thus, the sequence of probability measures {pn} satisfies the condition (*) required in the 
Gartner-Ellis theorem. A direct application of this theorem gives the bound 

limsup — logpn(/) < — inf (A(x) — A( 0 ))* 

n—>oo n x^I 

= — inf A*{x) — A( 0 ), 

xGl 


which immediately gives 


limsup - log;U„/„(/) < - inf A*(x). 
n^oo n x&I 


□ 


Remark F.5. If A{t) is differentiable, we can apply the Gartner-Ellis theorem to get a lower bound 
of the form 

liminf-log/r„/„(F) > - inf A*(x), 

n^oo n xGF 

for every open set F. However, it is easy to construct sub-convolutive sequences such that A{t) is 
not differentiable. One example is the sequence {pn} such that for each n, 




1 if j = 0 or re, 
0 otherwise. 


Theorem F.6. The functions {g^} converge uniformly to A* on [0,1]. 

Proof. We’ll show that {gn} converge pointwise to A* on [0,1]. Since < 7 * and A* are all continuous 
convex functions on a compact set. Lemma E.l implies that this pointwise convergence implies 
uniform convergence. 


Recall that a = inf„ ^ log//„(re), (3 = inf„ 
and define 


HogpniO), and inf„g'„(0) = A(0). Fix an x G (0,1), 


argmaxxf — gn{t) ■= tn- 
Glearly, g^ix) = xtn — gn{tn). Note that 


g*n{x) > xt-gn{t) 

= -fi'n(O) 

(a) 

> -51 ( 0 ) 


(282) 

(283) 

(284) 


where (a) follows by inequality (272). 
If ^ then we have 


xt — gn(i) < xt — (t + a) 

= —(1 — x)t — a 
< -(51(0) - a ) - a 
= -5i(0). 
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(285) 

(286) 
(287) 


This gives us that tn < " . Similarly, if t < ^ , then 


xt - gn{t) < xt- 13 

<W- 9i(0)) - P 
= -ffi(O). 

This gives us that tn > ■ We can thus conclude that for all n, 

'P-gi{0) gi{0)-a' 


tn £ 


I — X 


:=4. 


(288) 


Note that all we used to prove relation (288) is that gn is trapped between gi and max(/3, t + a) 
Since A also satisfies this, we have 


argmaxxt — A(t) G (289) 

We now restrict our attention to the compact interval Ix- Let gn be gn restricted to Ix- The 
convex functions gn converge pointwise to a continuous limit A, where A is A restricted to Ix- This 
convergence must therefore be uniform, which implies convergence of Pnix) to A*(x). Furthermore, 
relation (288) implies 5 ))(x) equals gn{x), and relation (289) gives A*(x) equals A*{x). Thus, gnp) 
converges pointwise to A*(-) on (0,1). 

We’ll now consider convergence at the boundary points. Let e > 0 be given. Choose a sub¬ 
sequence {gnt,} where Uk = 2*’. Using the condition in (269), it is clear that {gn^.} decrease 
monotonically and converge pointwise to A. Choose Kq large enough such that for all k > Ko, 

— log/r„,(0)-/3<e/2. (290) 

'^k 

Note that the left hand side is non-negative, and we need not use absolute values. Choose a Tq 
such that for all t < Tq, 

duKo (.t) - log gnKo (0) < e/2. (291) 

^Kq 

Now for all k > Kq and all t < Tq, the following holds: 

9nkit) - P < griKoit) - P < — log/^nKo(0) + e/2 - P < (-■ (292) 

Taking the limit in k, we get that for all t < Tq, 

A(t) -P <e, (293) 

this along with the lower bound A(t) > P gives that for all t <Tq, 

0 < A{t)- p< €. 

We also have 

A*(0) = sup — A(t) 

t 

= - lim A(t), 

t^ — OO 
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(294) 

(295) 











which must equal —f3. Since the limit of is also —/?, we have shown convergence of g* to A 

at t = 0 . 

To show convergence at t = 1, we follow a similar strategy. Let {gru.} be as before, and let 
e > 0 be given. We choose a Ki such that for all k > Ki, 

— log Huk ink) -a<el2. (296) 

Note that the left hand side is non-negative, and we need not use absolute values. We now choose 
a Ti such that for all t > Ti, 

dnji^it) - (t +—logUnK^iriKi)] < e/2. (297) 

^ \ riKi J 

Now for all k > Ki and all t > Ti, 

fl'nfc(i) - {t + a) < ffn [t] - {t + a) < log//nKi + e/2 - a < 6 . (298) 

riKi 

Taking the limit in k, we get that for all t > Ti, 

A{t) - {t + a) < e, 

this along with the lower bound A(t) > t + a gives that for all t > Ti 

0 < A(t) — (t + a) < e. 

From this, we conclude that A*(l) = sup^ t — A(t) = limi_ 5 .+oo t — A(t), must equal —a. Since the 
limit of 5 )^( 1 ) is also —a, we have shown convergence of 5 * to A at t = 1 . 

This shows that {< 7 ))} converges pointwise to A* on the compact interval [0,1]. As all the 
functions involved are continuous and convex, by Lemma E.l this convergence must also be uniform. 
This concludes the proof. □ 
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